2023-03-24 21:53:32,948 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-03-24 21:53:33,196 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5 2023-03-24 21:53:33,210 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-03-24 21:53:33,210 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/cluster_90ef77c7-dcec-9426-4d35-a9af4e7f4c97, deleteOnExit=true 2023-03-24 21:53:33,210 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-03-24 21:53:33,211 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/test.cache.data in system properties and HBase conf 2023-03-24 21:53:33,211 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/hadoop.tmp.dir in system properties and HBase conf 2023-03-24 21:53:33,212 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/hadoop.log.dir in system properties and HBase conf 2023-03-24 21:53:33,212 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/mapreduce.cluster.local.dir in system properties and HBase conf 2023-03-24 21:53:33,212 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-03-24 21:53:33,213 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-03-24 21:53:33,334 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-03-24 21:53:33,742 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-03-24 21:53:33,746 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-03-24 21:53:33,747 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-03-24 21:53:33,747 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-03-24 21:53:33,747 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-24 21:53:33,748 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-03-24 21:53:33,748 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-03-24 21:53:33,748 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-24 21:53:33,749 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-24 21:53:33,749 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-03-24 21:53:33,750 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/nfs.dump.dir in system properties and HBase conf 2023-03-24 21:53:33,750 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/java.io.tmpdir in system properties and HBase conf 2023-03-24 21:53:33,750 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-24 21:53:33,751 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-03-24 21:53:33,751 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-03-24 21:53:34,336 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-24 21:53:34,341 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-24 21:53:34,643 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-03-24 21:53:34,820 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-03-24 21:53:34,835 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-24 21:53:34,874 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-03-24 21:53:34,907 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/java.io.tmpdir/Jetty_localhost_35965_hdfs____.l6t8ao/webapp 2023-03-24 21:53:35,063 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35965 2023-03-24 21:53:35,078 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-24 21:53:35,079 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-24 21:53:35,658 WARN [Listener at localhost/37101] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-24 21:53:35,781 WARN [Listener at localhost/37101] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-03-24 21:53:35,799 WARN [Listener at localhost/37101] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-24 21:53:35,806 INFO [Listener at localhost/37101] log.Slf4jLog(67): jetty-6.1.26 2023-03-24 21:53:35,810 INFO [Listener at localhost/37101] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/java.io.tmpdir/Jetty_localhost_35585_datanode____.hy3npl/webapp 2023-03-24 21:53:35,916 INFO [Listener at localhost/37101] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35585 2023-03-24 21:53:36,287 WARN [Listener at localhost/39907] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-24 21:53:36,831 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc8fa3f4ec40bf4d3: Processing first storage report for DS-94767ee5-d52d-4af7-90c6-13bb97590b6f from datanode 21e86932-ee7d-48f1-8eb1-077868519d45 2023-03-24 21:53:36,832 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc8fa3f4ec40bf4d3: from storage DS-94767ee5-d52d-4af7-90c6-13bb97590b6f node DatanodeRegistration(127.0.0.1:46017, datanodeUuid=21e86932-ee7d-48f1-8eb1-077868519d45, infoPort=37685, infoSecurePort=0, ipcPort=39907, storageInfo=lv=-57;cid=testClusterID;nsid=1255930391;c=1679694814421), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-03-24 21:53:36,832 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc8fa3f4ec40bf4d3: Processing first storage report for DS-06c5bfd5-b1f9-4f48-b3f1-43f3751d7898 from datanode 21e86932-ee7d-48f1-8eb1-077868519d45 2023-03-24 21:53:36,833 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc8fa3f4ec40bf4d3: from storage DS-06c5bfd5-b1f9-4f48-b3f1-43f3751d7898 node DatanodeRegistration(127.0.0.1:46017, datanodeUuid=21e86932-ee7d-48f1-8eb1-077868519d45, infoPort=37685, infoSecurePort=0, ipcPort=39907, storageInfo=lv=-57;cid=testClusterID;nsid=1255930391;c=1679694814421), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-03-24 21:53:36,862 DEBUG [Listener at localhost/39907] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5 2023-03-24 21:53:36,931 INFO [Listener at localhost/39907] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/cluster_90ef77c7-dcec-9426-4d35-a9af4e7f4c97/zookeeper_0, clientPort=54680, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/cluster_90ef77c7-dcec-9426-4d35-a9af4e7f4c97/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/cluster_90ef77c7-dcec-9426-4d35-a9af4e7f4c97/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-03-24 21:53:36,949 INFO [Listener at localhost/39907] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=54680 2023-03-24 21:53:36,957 INFO [Listener at localhost/39907] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-24 21:53:36,959 INFO [Listener at localhost/39907] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-24 21:53:37,663 INFO [Listener at localhost/39907] util.FSUtils(471): Created version file at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b with version=8 2023-03-24 21:53:37,663 INFO [Listener at localhost/39907] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/hbase-staging 2023-03-24 21:53:38,003 INFO [Listener at localhost/39907] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-03-24 21:53:38,536 INFO [Listener at localhost/39907] client.ConnectionUtils(127): master/jenkins-hbase9:0 server-side Connection retries=3 2023-03-24 21:53:38,567 INFO [Listener at localhost/39907] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-24 21:53:38,567 INFO [Listener at localhost/39907] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-24 21:53:38,567 INFO [Listener at localhost/39907] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-24 21:53:38,568 INFO [Listener at localhost/39907] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-24 21:53:38,568 INFO [Listener at localhost/39907] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-24 21:53:38,727 INFO [Listener at localhost/39907] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-03-24 21:53:38,811 DEBUG [Listener at localhost/39907] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-03-24 21:53:38,905 INFO [Listener at localhost/39907] ipc.NettyRpcServer(120): Bind to /172.31.2.10:46449 2023-03-24 21:53:38,915 INFO [Listener at localhost/39907] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-24 21:53:38,917 INFO [Listener at localhost/39907] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-24 21:53:38,940 INFO [Listener at localhost/39907] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46449 connecting to ZooKeeper ensemble=127.0.0.1:54680 2023-03-24 21:53:38,984 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:464490x0, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-24 21:53:38,987 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:46449-0x1011885c6830000 connected 2023-03-24 21:53:39,012 DEBUG [Listener at localhost/39907] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-24 21:53:39,013 DEBUG [Listener at localhost/39907] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-24 21:53:39,017 DEBUG [Listener at localhost/39907] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-24 21:53:39,024 DEBUG [Listener at localhost/39907] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46449 2023-03-24 21:53:39,025 DEBUG [Listener at localhost/39907] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46449 2023-03-24 21:53:39,025 DEBUG [Listener at localhost/39907] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46449 2023-03-24 21:53:39,026 DEBUG [Listener at localhost/39907] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46449 2023-03-24 21:53:39,026 DEBUG [Listener at localhost/39907] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46449 2023-03-24 21:53:39,033 INFO [Listener at localhost/39907] master.HMaster(444): hbase.rootdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b, hbase.cluster.distributed=false 2023-03-24 21:53:39,103 INFO [Listener at localhost/39907] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=3 2023-03-24 21:53:39,104 INFO [Listener at localhost/39907] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-24 21:53:39,104 INFO [Listener at localhost/39907] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-24 21:53:39,104 INFO [Listener at localhost/39907] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-24 21:53:39,104 INFO [Listener at localhost/39907] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-24 21:53:39,104 INFO [Listener at localhost/39907] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-24 21:53:39,109 INFO [Listener at localhost/39907] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-03-24 21:53:39,112 INFO [Listener at localhost/39907] ipc.NettyRpcServer(120): Bind to /172.31.2.10:41807 2023-03-24 21:53:39,114 INFO [Listener at localhost/39907] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-03-24 21:53:39,120 DEBUG [Listener at localhost/39907] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-03-24 21:53:39,121 INFO [Listener at localhost/39907] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-24 21:53:39,125 INFO [Listener at localhost/39907] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-24 21:53:39,127 INFO [Listener at localhost/39907] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41807 connecting to ZooKeeper ensemble=127.0.0.1:54680 2023-03-24 21:53:39,130 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:418070x0, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-24 21:53:39,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41807-0x1011885c6830001 connected 2023-03-24 21:53:39,131 DEBUG [Listener at localhost/39907] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-24 21:53:39,133 DEBUG [Listener at localhost/39907] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-24 21:53:39,133 DEBUG [Listener at localhost/39907] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-24 21:53:39,134 DEBUG [Listener at localhost/39907] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41807 2023-03-24 21:53:39,134 DEBUG [Listener at localhost/39907] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41807 2023-03-24 21:53:39,136 DEBUG [Listener at localhost/39907] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41807 2023-03-24 21:53:39,136 DEBUG [Listener at localhost/39907] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41807 2023-03-24 21:53:39,137 DEBUG [Listener at localhost/39907] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41807 2023-03-24 21:53:39,139 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase9.apache.org,46449,1679694817811 2023-03-24 21:53:39,148 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-24 21:53:39,149 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase9.apache.org,46449,1679694817811 2023-03-24 21:53:39,170 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-24 21:53:39,170 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-24 21:53:39,170 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-24 21:53:39,171 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-24 21:53:39,173 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase9.apache.org,46449,1679694817811 from backup master directory 2023-03-24 21:53:39,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-24 21:53:39,178 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase9.apache.org,46449,1679694817811 2023-03-24 21:53:39,178 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-24 21:53:39,178 WARN [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-24 21:53:39,178 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase9.apache.org,46449,1679694817811 2023-03-24 21:53:39,181 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-03-24 21:53:39,183 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-03-24 21:53:39,266 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/hbase.id with ID: e5ed0dba-d7eb-4e49-bf2d-39f2daf201ff 2023-03-24 21:53:39,305 INFO [master/jenkins-hbase9:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-24 21:53:39,319 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-24 21:53:39,355 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x5ab37b78 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:53:39,386 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4aa8f3a0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:53:39,409 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-03-24 21:53:39,411 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-03-24 21:53:39,427 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-03-24 21:53:39,428 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-03-24 21:53:39,429 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-24 21:53:39,433 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-24 21:53:39,434 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-24 21:53:39,467 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/data/master/store-tmp 2023-03-24 21:53:39,492 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:53:39,492 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-24 21:53:39,492 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-24 21:53:39,492 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-24 21:53:39,492 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-24 21:53:39,492 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-24 21:53:39,493 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-24 21:53:39,493 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-24 21:53:39,494 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/WALs/jenkins-hbase9.apache.org,46449,1679694817811 2023-03-24 21:53:39,517 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C46449%2C1679694817811, suffix=, logDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/WALs/jenkins-hbase9.apache.org,46449,1679694817811, archiveDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/oldWALs, maxLogs=10 2023-03-24 21:53:39,578 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46017,DS-94767ee5-d52d-4af7-90c6-13bb97590b6f,DISK] 2023-03-24 21:53:39,586 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-03-24 21:53:39,648 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/WALs/jenkins-hbase9.apache.org,46449,1679694817811/jenkins-hbase9.apache.org%2C46449%2C1679694817811.1679694819526 2023-03-24 21:53:39,648 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46017,DS-94767ee5-d52d-4af7-90c6-13bb97590b6f,DISK]] 2023-03-24 21:53:39,649 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:53:39,649 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:53:39,653 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-03-24 21:53:39,654 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-03-24 21:53:39,718 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-03-24 21:53:39,727 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-03-24 21:53:39,752 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-03-24 21:53:39,764 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:53:39,772 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-24 21:53:39,773 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-24 21:53:39,790 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-03-24 21:53:39,794 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:53:39,797 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61414476, jitterRate=-0.08485299348831177}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:53:39,797 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-24 21:53:39,799 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-03-24 21:53:39,818 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-03-24 21:53:39,818 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-03-24 21:53:39,820 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-03-24 21:53:39,822 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-03-24 21:53:39,854 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-03-24 21:53:39,854 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-03-24 21:53:39,880 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-03-24 21:53:39,886 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-03-24 21:53:39,909 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-03-24 21:53:39,913 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-03-24 21:53:39,915 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-03-24 21:53:39,920 INFO [master/jenkins-hbase9:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-03-24 21:53:39,924 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-03-24 21:53:39,927 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-24 21:53:39,929 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-03-24 21:53:39,929 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-03-24 21:53:39,942 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-03-24 21:53:39,946 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-24 21:53:39,946 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-24 21:53:39,946 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-24 21:53:39,947 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase9.apache.org,46449,1679694817811, sessionid=0x1011885c6830000, setting cluster-up flag (Was=false) 2023-03-24 21:53:39,962 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-24 21:53:39,968 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-03-24 21:53:39,969 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,46449,1679694817811 2023-03-24 21:53:39,975 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-24 21:53:39,981 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-03-24 21:53:39,982 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,46449,1679694817811 2023-03-24 21:53:39,985 WARN [master/jenkins-hbase9:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.hbase-snapshot/.tmp 2023-03-24 21:53:40,041 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(952): ClusterId : e5ed0dba-d7eb-4e49-bf2d-39f2daf201ff 2023-03-24 21:53:40,045 DEBUG [RS:0;jenkins-hbase9:41807] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-03-24 21:53:40,050 DEBUG [RS:0;jenkins-hbase9:41807] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-03-24 21:53:40,051 DEBUG [RS:0;jenkins-hbase9:41807] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-03-24 21:53:40,054 DEBUG [RS:0;jenkins-hbase9:41807] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-03-24 21:53:40,055 DEBUG [RS:0;jenkins-hbase9:41807] zookeeper.ReadOnlyZKClient(139): Connect 0x66aaf288 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:53:40,060 DEBUG [RS:0;jenkins-hbase9:41807] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@686d002d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:53:40,060 DEBUG [RS:0;jenkins-hbase9:41807] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c5299f8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-03-24 21:53:40,083 DEBUG [RS:0;jenkins-hbase9:41807] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase9:41807 2023-03-24 21:53:40,086 INFO [RS:0;jenkins-hbase9:41807] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-03-24 21:53:40,087 INFO [RS:0;jenkins-hbase9:41807] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-03-24 21:53:40,087 DEBUG [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1023): About to register with Master. 2023-03-24 21:53:40,089 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase9.apache.org,46449,1679694817811 with isa=jenkins-hbase9.apache.org/172.31.2.10:41807, startcode=1679694819103 2023-03-24 21:53:40,098 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-03-24 21:53:40,106 DEBUG [RS:0;jenkins-hbase9:41807] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-03-24 21:53:40,110 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-03-24 21:53:40,110 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-03-24 21:53:40,110 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-03-24 21:53:40,110 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-03-24 21:53:40,110 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase9:0, corePoolSize=10, maxPoolSize=10 2023-03-24 21:53:40,110 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-03-24 21:53:40,111 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-03-24 21:53:40,111 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-03-24 21:53:40,112 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1679694850112 2023-03-24 21:53:40,115 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-03-24 21:53:40,118 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-03-24 21:53:40,118 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-03-24 21:53:40,123 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-24 21:53:40,127 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-03-24 21:53:40,134 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-03-24 21:53:40,134 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-03-24 21:53:40,134 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-03-24 21:53:40,134 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-03-24 21:53:40,135 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:40,136 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-03-24 21:53:40,138 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-03-24 21:53:40,138 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-03-24 21:53:40,140 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-03-24 21:53:40,141 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-03-24 21:53:40,142 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1679694820142,5,FailOnTimeoutGroup] 2023-03-24 21:53:40,143 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1679694820143,5,FailOnTimeoutGroup] 2023-03-24 21:53:40,143 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:40,143 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-03-24 21:53:40,144 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:40,144 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:40,172 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:57853, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-03-24 21:53:40,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:40,197 DEBUG [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b 2023-03-24 21:53:40,197 DEBUG [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:37101 2023-03-24 21:53:40,197 DEBUG [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-03-24 21:53:40,203 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-24 21:53:40,204 DEBUG [RS:0;jenkins-hbase9:41807] zookeeper.ZKUtil(162): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:40,204 WARN [RS:0;jenkins-hbase9:41807] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-24 21:53:40,205 INFO [RS:0;jenkins-hbase9:41807] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-24 21:53:40,205 DEBUG [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1947): logDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/WALs/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:40,206 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,41807,1679694819103] 2023-03-24 21:53:40,214 DEBUG [RS:0;jenkins-hbase9:41807] zookeeper.ZKUtil(162): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:40,223 DEBUG [RS:0;jenkins-hbase9:41807] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-03-24 21:53:40,231 INFO [RS:0;jenkins-hbase9:41807] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-03-24 21:53:40,249 INFO [RS:0;jenkins-hbase9:41807] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-03-24 21:53:40,252 INFO [RS:0;jenkins-hbase9:41807] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:53:40,252 INFO [RS:0;jenkins-hbase9:41807] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:40,254 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-03-24 21:53:40,261 INFO [RS:0;jenkins-hbase9:41807] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-03-24 21:53:40,261 DEBUG [RS:0;jenkins-hbase9:41807] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-03-24 21:53:40,261 DEBUG [RS:0;jenkins-hbase9:41807] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-03-24 21:53:40,261 DEBUG [RS:0;jenkins-hbase9:41807] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-03-24 21:53:40,262 DEBUG [RS:0;jenkins-hbase9:41807] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-03-24 21:53:40,262 DEBUG [RS:0;jenkins-hbase9:41807] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-03-24 21:53:40,262 DEBUG [RS:0;jenkins-hbase9:41807] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-03-24 21:53:40,262 DEBUG [RS:0;jenkins-hbase9:41807] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-03-24 21:53:40,262 DEBUG [RS:0;jenkins-hbase9:41807] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-03-24 21:53:40,262 DEBUG [RS:0;jenkins-hbase9:41807] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-03-24 21:53:40,262 DEBUG [RS:0;jenkins-hbase9:41807] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-03-24 21:53:40,263 INFO [RS:0;jenkins-hbase9:41807] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:40,263 INFO [RS:0;jenkins-hbase9:41807] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:40,263 INFO [RS:0;jenkins-hbase9:41807] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:40,280 INFO [RS:0;jenkins-hbase9:41807] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-03-24 21:53:40,283 INFO [RS:0;jenkins-hbase9:41807] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,41807,1679694819103-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:40,299 INFO [RS:0;jenkins-hbase9:41807] regionserver.Replication(203): jenkins-hbase9.apache.org,41807,1679694819103 started 2023-03-24 21:53:40,299 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1638): Serving as jenkins-hbase9.apache.org,41807,1679694819103, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:41807, sessionid=0x1011885c6830001 2023-03-24 21:53:40,299 DEBUG [RS:0;jenkins-hbase9:41807] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-03-24 21:53:40,299 DEBUG [RS:0;jenkins-hbase9:41807] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:40,299 DEBUG [RS:0;jenkins-hbase9:41807] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,41807,1679694819103' 2023-03-24 21:53:40,299 DEBUG [RS:0;jenkins-hbase9:41807] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:40,300 DEBUG [RS:0;jenkins-hbase9:41807] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:40,301 DEBUG [RS:0;jenkins-hbase9:41807] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-03-24 21:53:40,301 DEBUG [RS:0;jenkins-hbase9:41807] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-03-24 21:53:40,301 DEBUG [RS:0;jenkins-hbase9:41807] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:40,301 DEBUG [RS:0;jenkins-hbase9:41807] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,41807,1679694819103' 2023-03-24 21:53:40,301 DEBUG [RS:0;jenkins-hbase9:41807] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-24 21:53:40,302 DEBUG [RS:0;jenkins-hbase9:41807] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-24 21:53:40,302 DEBUG [RS:0;jenkins-hbase9:41807] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-03-24 21:53:40,302 INFO [RS:0;jenkins-hbase9:41807] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-03-24 21:53:40,302 INFO [RS:0;jenkins-hbase9:41807] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-03-24 21:53:40,416 INFO [RS:0;jenkins-hbase9:41807] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C41807%2C1679694819103, suffix=, logDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/WALs/jenkins-hbase9.apache.org,41807,1679694819103, archiveDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/oldWALs, maxLogs=32 2023-03-24 21:53:40,436 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46017,DS-94767ee5-d52d-4af7-90c6-13bb97590b6f,DISK] 2023-03-24 21:53:40,440 INFO [RS:0;jenkins-hbase9:41807] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/WALs/jenkins-hbase9.apache.org,41807,1679694819103/jenkins-hbase9.apache.org%2C41807%2C1679694819103.1679694820419 2023-03-24 21:53:40,441 DEBUG [RS:0;jenkins-hbase9:41807] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46017,DS-94767ee5-d52d-4af7-90c6-13bb97590b6f,DISK]] 2023-03-24 21:53:40,563 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-24 21:53:40,564 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-24 21:53:40,565 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b 2023-03-24 21:53:40,578 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:53:40,581 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-24 21:53:40,583 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/info 2023-03-24 21:53:40,585 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-24 21:53:40,586 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:53:40,586 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-24 21:53:40,588 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/rep_barrier 2023-03-24 21:53:40,589 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-24 21:53:40,590 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:53:40,590 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-24 21:53:40,592 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/table 2023-03-24 21:53:40,593 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-24 21:53:40,594 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:53:40,595 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740 2023-03-24 21:53:40,596 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740 2023-03-24 21:53:40,600 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-24 21:53:40,602 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-24 21:53:40,605 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:53:40,606 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61102062, jitterRate=-0.08950832486152649}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-24 21:53:40,606 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-24 21:53:40,606 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-24 21:53:40,606 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-24 21:53:40,607 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-24 21:53:40,607 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-24 21:53:40,607 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-24 21:53:40,608 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-24 21:53:40,608 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-24 21:53:40,614 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-03-24 21:53:40,614 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-03-24 21:53:40,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-03-24 21:53:40,635 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-03-24 21:53:40,638 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-03-24 21:53:40,789 DEBUG [jenkins-hbase9:46449] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-03-24 21:53:40,794 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,41807,1679694819103, state=OPENING 2023-03-24 21:53:40,803 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-03-24 21:53:40,805 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-24 21:53:40,806 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-24 21:53:40,809 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:53:40,994 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:40,996 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-24 21:53:41,000 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:52730, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-24 21:53:41,012 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-03-24 21:53:41,013 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-24 21:53:41,016 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C41807%2C1679694819103.meta, suffix=.meta, logDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/WALs/jenkins-hbase9.apache.org,41807,1679694819103, archiveDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/oldWALs, maxLogs=32 2023-03-24 21:53:41,033 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46017,DS-94767ee5-d52d-4af7-90c6-13bb97590b6f,DISK] 2023-03-24 21:53:41,038 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/WALs/jenkins-hbase9.apache.org,41807,1679694819103/jenkins-hbase9.apache.org%2C41807%2C1679694819103.meta.1679694821017.meta 2023-03-24 21:53:41,038 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46017,DS-94767ee5-d52d-4af7-90c6-13bb97590b6f,DISK]] 2023-03-24 21:53:41,038 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:53:41,040 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-03-24 21:53:41,058 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-03-24 21:53:41,064 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-03-24 21:53:41,070 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-03-24 21:53:41,070 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:53:41,070 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-03-24 21:53:41,071 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-03-24 21:53:41,073 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-24 21:53:41,075 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/info 2023-03-24 21:53:41,075 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/info 2023-03-24 21:53:41,076 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-24 21:53:41,077 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:53:41,077 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-24 21:53:41,079 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/rep_barrier 2023-03-24 21:53:41,079 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/rep_barrier 2023-03-24 21:53:41,080 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-24 21:53:41,081 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:53:41,081 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-24 21:53:41,083 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/table 2023-03-24 21:53:41,083 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/table 2023-03-24 21:53:41,083 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-24 21:53:41,084 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:53:41,086 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740 2023-03-24 21:53:41,088 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740 2023-03-24 21:53:41,091 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-24 21:53:41,093 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-24 21:53:41,094 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61533759, jitterRate=-0.08307553827762604}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-24 21:53:41,094 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-24 21:53:41,104 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1679694820987 2023-03-24 21:53:41,120 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-03-24 21:53:41,121 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-03-24 21:53:41,122 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,41807,1679694819103, state=OPEN 2023-03-24 21:53:41,125 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-03-24 21:53:41,125 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-24 21:53:41,128 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-03-24 21:53:41,129 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,41807,1679694819103 in 316 msec 2023-03-24 21:53:41,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-03-24 21:53:41,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 506 msec 2023-03-24 21:53:41,138 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.1090 sec 2023-03-24 21:53:41,138 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1679694821138, completionTime=-1 2023-03-24 21:53:41,139 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-03-24 21:53:41,139 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-03-24 21:53:41,203 DEBUG [hconnection-0x5ea55c48-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:53:41,208 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:52740, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:53:41,226 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-03-24 21:53:41,226 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1679694881226 2023-03-24 21:53:41,226 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1679694941226 2023-03-24 21:53:41,226 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 87 msec 2023-03-24 21:53:41,249 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,46449,1679694817811-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:41,249 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,46449,1679694817811-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:41,249 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,46449,1679694817811-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:41,251 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase9:46449, period=300000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:41,251 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-03-24 21:53:41,257 DEBUG [master/jenkins-hbase9:0.Chore.1] janitor.CatalogJanitor(175): 2023-03-24 21:53:41,265 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-03-24 21:53:41,266 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-24 21:53:41,275 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-03-24 21:53:41,277 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:53:41,279 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:53:41,299 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1 2023-03-24 21:53:41,301 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1 empty. 2023-03-24 21:53:41,302 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1 2023-03-24 21:53:41,302 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-03-24 21:53:41,322 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-03-24 21:53:41,324 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 35c7396e04016558d3f104322a61b9d1, NAME => 'hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:53:41,336 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:53:41,336 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 35c7396e04016558d3f104322a61b9d1, disabling compactions & flushes 2023-03-24 21:53:41,336 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. 2023-03-24 21:53:41,337 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. 2023-03-24 21:53:41,337 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. after waiting 0 ms 2023-03-24 21:53:41,337 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. 2023-03-24 21:53:41,337 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. 2023-03-24 21:53:41,337 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 35c7396e04016558d3f104322a61b9d1: 2023-03-24 21:53:41,341 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:53:41,358 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1679694821344"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694821344"}]},"ts":"1679694821344"} 2023-03-24 21:53:41,385 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-24 21:53:41,387 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:53:41,392 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694821388"}]},"ts":"1679694821388"} 2023-03-24 21:53:41,396 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-03-24 21:53:41,405 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=35c7396e04016558d3f104322a61b9d1, ASSIGN}] 2023-03-24 21:53:41,407 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=35c7396e04016558d3f104322a61b9d1, ASSIGN 2023-03-24 21:53:41,409 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=35c7396e04016558d3f104322a61b9d1, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:53:41,560 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=35c7396e04016558d3f104322a61b9d1, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:41,561 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1679694821560"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694821560"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694821560"}]},"ts":"1679694821560"} 2023-03-24 21:53:41,567 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 35c7396e04016558d3f104322a61b9d1, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:53:41,727 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. 2023-03-24 21:53:41,727 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 35c7396e04016558d3f104322a61b9d1, NAME => 'hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:53:41,728 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 35c7396e04016558d3f104322a61b9d1 2023-03-24 21:53:41,729 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:53:41,729 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 35c7396e04016558d3f104322a61b9d1 2023-03-24 21:53:41,729 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 35c7396e04016558d3f104322a61b9d1 2023-03-24 21:53:41,731 INFO [StoreOpener-35c7396e04016558d3f104322a61b9d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 35c7396e04016558d3f104322a61b9d1 2023-03-24 21:53:41,733 DEBUG [StoreOpener-35c7396e04016558d3f104322a61b9d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1/info 2023-03-24 21:53:41,733 DEBUG [StoreOpener-35c7396e04016558d3f104322a61b9d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1/info 2023-03-24 21:53:41,734 INFO [StoreOpener-35c7396e04016558d3f104322a61b9d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35c7396e04016558d3f104322a61b9d1 columnFamilyName info 2023-03-24 21:53:41,734 INFO [StoreOpener-35c7396e04016558d3f104322a61b9d1-1] regionserver.HStore(310): Store=35c7396e04016558d3f104322a61b9d1/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:53:41,736 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1 2023-03-24 21:53:41,737 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1 2023-03-24 21:53:41,741 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 35c7396e04016558d3f104322a61b9d1 2023-03-24 21:53:41,745 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:53:41,746 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 35c7396e04016558d3f104322a61b9d1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69387447, jitterRate=0.033953532576560974}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:53:41,746 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 35c7396e04016558d3f104322a61b9d1: 2023-03-24 21:53:41,748 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1., pid=6, masterSystemTime=1679694821721 2023-03-24 21:53:41,752 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. 2023-03-24 21:53:41,752 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. 2023-03-24 21:53:41,754 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=35c7396e04016558d3f104322a61b9d1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:41,754 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1679694821753"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694821753"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694821753"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694821753"}]},"ts":"1679694821753"} 2023-03-24 21:53:41,761 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-03-24 21:53:41,761 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 35c7396e04016558d3f104322a61b9d1, server=jenkins-hbase9.apache.org,41807,1679694819103 in 191 msec 2023-03-24 21:53:41,765 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-03-24 21:53:41,766 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=35c7396e04016558d3f104322a61b9d1, ASSIGN in 356 msec 2023-03-24 21:53:41,766 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:53:41,767 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694821767"}]},"ts":"1679694821767"} 2023-03-24 21:53:41,770 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-03-24 21:53:41,774 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:53:41,776 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 507 msec 2023-03-24 21:53:41,777 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-03-24 21:53:41,779 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-03-24 21:53:41,779 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-24 21:53:41,822 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-03-24 21:53:41,839 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-24 21:53:41,845 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 34 msec 2023-03-24 21:53:41,856 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-03-24 21:53:41,867 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-24 21:53:41,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-03-24 21:53:41,883 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-03-24 21:53:41,885 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-03-24 21:53:41,886 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.706sec 2023-03-24 21:53:41,887 INFO [master/jenkins-hbase9:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-03-24 21:53:41,889 INFO [master/jenkins-hbase9:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-03-24 21:53:41,889 INFO [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-03-24 21:53:41,890 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,46449,1679694817811-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-03-24 21:53:41,890 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,46449,1679694817811-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-03-24 21:53:41,901 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-03-24 21:53:41,947 DEBUG [Listener at localhost/39907] zookeeper.ReadOnlyZKClient(139): Connect 0x0672b468 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:53:41,952 DEBUG [Listener at localhost/39907] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@561bf69b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:53:41,966 DEBUG [hconnection-0x78f916d2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:53:41,981 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:52752, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:53:41,991 INFO [Listener at localhost/39907] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase9.apache.org,46449,1679694817811 2023-03-24 21:53:41,992 DEBUG [Listener at localhost/39907] zookeeper.ReadOnlyZKClient(139): Connect 0x33fc1e18 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:53:41,997 DEBUG [Listener at localhost/39907] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b586bb5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:53:41,997 INFO [Listener at localhost/39907] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-24 21:53:42,020 INFO [Listener at localhost/39907] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=211, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=291, ProcessCount=168, AvailableMemoryMB=8991 2023-03-24 21:53:42,033 DEBUG [Listener at localhost/39907] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:53:42,037 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:48246, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:53:42,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-24 21:53:42,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:42,055 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:53:42,058 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:53:42,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-03-24 21:53:42,062 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:42,063 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da empty. 2023-03-24 21:53:42,065 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:42,066 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-03-24 21:53:42,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-24 21:53:42,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-24 21:53:42,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-24 21:53:42,484 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-24 21:53:42,486 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => b8b55d7b0784a827760bb4f27b0708da, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:53:42,498 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:53:42,498 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing b8b55d7b0784a827760bb4f27b0708da, disabling compactions & flushes 2023-03-24 21:53:42,498 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:42,498 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:42,498 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. after waiting 0 ms 2023-03-24 21:53:42,498 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:42,498 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:42,498 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:42,502 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:53:42,504 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679694822503"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694822503"}]},"ts":"1679694822503"} 2023-03-24 21:53:42,506 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-24 21:53:42,508 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:53:42,508 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694822508"}]},"ts":"1679694822508"} 2023-03-24 21:53:42,510 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-03-24 21:53:42,514 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=b8b55d7b0784a827760bb4f27b0708da, ASSIGN}] 2023-03-24 21:53:42,516 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=b8b55d7b0784a827760bb4f27b0708da, ASSIGN 2023-03-24 21:53:42,517 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=b8b55d7b0784a827760bb4f27b0708da, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:53:42,668 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=b8b55d7b0784a827760bb4f27b0708da, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:42,669 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679694822668"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694822668"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694822668"}]},"ts":"1679694822668"} 2023-03-24 21:53:42,672 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure b8b55d7b0784a827760bb4f27b0708da, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:53:42,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-24 21:53:42,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:42,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b8b55d7b0784a827760bb4f27b0708da, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:53:42,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:42,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:53:42,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:42,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:42,835 INFO [StoreOpener-b8b55d7b0784a827760bb4f27b0708da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:42,837 DEBUG [StoreOpener-b8b55d7b0784a827760bb4f27b0708da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1 2023-03-24 21:53:42,837 DEBUG [StoreOpener-b8b55d7b0784a827760bb4f27b0708da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1 2023-03-24 21:53:42,840 INFO [StoreOpener-b8b55d7b0784a827760bb4f27b0708da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b8b55d7b0784a827760bb4f27b0708da columnFamilyName family1 2023-03-24 21:53:42,842 INFO [StoreOpener-b8b55d7b0784a827760bb4f27b0708da-1] regionserver.HStore(310): Store=b8b55d7b0784a827760bb4f27b0708da/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:53:42,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:42,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:42,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:42,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:53:42,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened b8b55d7b0784a827760bb4f27b0708da; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64145671, jitterRate=-0.04415501654148102}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:53:42,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:42,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da., pid=11, masterSystemTime=1679694822825 2023-03-24 21:53:42,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:42,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:42,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=b8b55d7b0784a827760bb4f27b0708da, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:42,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679694822858"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694822858"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694822858"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694822858"}]},"ts":"1679694822858"} 2023-03-24 21:53:42,865 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-03-24 21:53:42,865 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure b8b55d7b0784a827760bb4f27b0708da, server=jenkins-hbase9.apache.org,41807,1679694819103 in 190 msec 2023-03-24 21:53:42,868 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-03-24 21:53:42,869 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=b8b55d7b0784a827760bb4f27b0708da, ASSIGN in 351 msec 2023-03-24 21:53:42,870 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:53:42,870 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694822870"}]},"ts":"1679694822870"} 2023-03-24 21:53:42,872 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-03-24 21:53:42,875 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:53:42,877 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 824 msec 2023-03-24 21:53:43,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-24 21:53:43,184 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-03-24 21:53:43,202 DEBUG [hconnection-0x7aa6bd5c-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:53:43,206 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:52754, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:53:43,366 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:43,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,373 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:43,373 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:43,374 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-24 21:53:43,374 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-24 21:53:43,374 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,374 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,377 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:43,377 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:43,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:43,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:43,377 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:43,377 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:43,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:43,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,379 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,380 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,381 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-24 21:53:43,381 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:43,381 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-24 21:53:43,382 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:43,382 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:43,382 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:43,383 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. started... 2023-03-24 21:53:43,384 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b8b55d7b0784a827760bb4f27b0708da 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-24 21:53:43,925 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d45b5df239b34e5bbbca4ef00934c63f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130d45b5df239b34e5bbbca4ef00934c63f 2023-03-24 21:53:43,944 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/17e04a222db94f9fa7f4e1c649c9ab4d 2023-03-24 21:53:43,955 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/17e04a222db94f9fa7f4e1c649c9ab4d as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/17e04a222db94f9fa7f4e1c649c9ab4d 2023-03-24 21:53:43,966 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/17e04a222db94f9fa7f4e1c649c9ab4d, entries=1000, sequenceid=5, filesize=175.8 K 2023-03-24 21:53:43,969 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for b8b55d7b0784a827760bb4f27b0708da in 585ms, sequenceid=5, compaction requested=false 2023-03-24 21:53:43,970 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:43,973 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:43,973 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:43,973 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:43,974 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:43,974 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:43,974 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-24 21:53:43,974 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-24 21:53:43,980 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,981 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:43,981 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:43,981 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:43,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:43,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:43,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:43,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:43,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:43,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:43,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-24 21:53:43,984 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-24 21:53:43,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7940a42e[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:43,984 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,986 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,987 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-24 21:53:43,987 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-24 21:53:43,987 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:43,987 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:43,987 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:43,990 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-24 21:53:43,990 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:43,990 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:43,990 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-24 21:53:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:43,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:44,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-24 21:53:44,001 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-24 21:53:44,001 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:44,001 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:44,001 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,001 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:44,003 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,004 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:44,003 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:44,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:44,005 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:44,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:44,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:44,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:44,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,012 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:44,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,027 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,027 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,027 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,027 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,027 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,027 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,027 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:44,027 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:44,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:44,027 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:44,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:44,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:44,030 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:44,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:44,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:44,034 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:44,034 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:44,134 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:44,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:44,157 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:44,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:44,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,174 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:44,174 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:44,175 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-24 21:53:44,175 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-24 21:53:44,176 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,176 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,178 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,178 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:44,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:44,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:44,179 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,179 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,180 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,181 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-24 21:53:44,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,183 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-24 21:53:44,183 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:44,184 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-24 21:53:44,185 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:44,185 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:44,185 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:44,185 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. started... 2023-03-24 21:53:44,185 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b8b55d7b0784a827760bb4f27b0708da 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-24 21:53:44,236 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511288e9529b271f14d7fa8684e6ec8f3579a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511288e9529b271f14d7fa8684e6ec8f3579a 2023-03-24 21:53:44,257 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/5d589de0e0094f18a8a6a6f9de26661a 2023-03-24 21:53:44,273 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/5d589de0e0094f18a8a6a6f9de26661a as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/5d589de0e0094f18a8a6a6f9de26661a 2023-03-24 21:53:44,286 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/5d589de0e0094f18a8a6a6f9de26661a, entries=1, sequenceid=9, filesize=5.0 K 2023-03-24 21:53:44,288 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for b8b55d7b0784a827760bb4f27b0708da in 103ms, sequenceid=9, compaction requested=false 2023-03-24 21:53:44,289 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:44,289 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:44,289 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:44,289 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:44,289 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,290 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-24 21:53:44,290 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-24 21:53:44,292 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,292 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:44,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:44,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:44,293 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,293 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:44,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:44,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:44,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-24 21:53:44,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6323b3e8[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:44,296 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-24 21:53:44,296 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,299 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,299 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-24 21:53:44,299 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-24 21:53:44,299 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,299 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:44,299 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:44,302 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-24 21:53:44,302 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,302 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:44,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:44,302 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-24 21:53:44,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:44,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:44,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:44,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:44,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:44,306 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-24 21:53:44,306 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-24 21:53:44,306 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:44,306 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:44,306 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,306 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:44,311 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,311 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:44,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,311 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:44,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:44,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:44,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:44,312 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:44,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:44,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,314 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,324 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,324 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,324 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:44,324 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:44,325 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:44,325 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,325 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:44,325 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,325 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,325 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:44,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:44,326 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-24 21:53:44,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:44,326 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:44,326 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:44,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:44,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:44,426 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:44,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:44,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:44,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:44,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,442 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:44,442 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:44,442 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-24 21:53:44,442 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-24 21:53:44,443 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,443 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,447 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:44,447 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:44,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:44,448 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,448 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:44,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,448 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:44,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,450 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,450 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-24 21:53:44,451 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,451 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-24 21:53:44,451 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:44,451 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-24 21:53:44,452 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:44,452 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:44,452 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:44,452 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. started... 2023-03-24 21:53:44,452 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b8b55d7b0784a827760bb4f27b0708da 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:44,504 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201115b300fa5fa4c37a98127a95ffb54de to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201115b300fa5fa4c37a98127a95ffb54de 2023-03-24 21:53:44,541 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/577435a1966f43449ea06e6a5f3e10f1 2023-03-24 21:53:44,553 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/577435a1966f43449ea06e6a5f3e10f1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/577435a1966f43449ea06e6a5f3e10f1 2023-03-24 21:53:44,572 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/577435a1966f43449ea06e6a5f3e10f1, entries=1, sequenceid=13, filesize=5.0 K 2023-03-24 21:53:44,574 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b8b55d7b0784a827760bb4f27b0708da in 122ms, sequenceid=13, compaction requested=true 2023-03-24 21:53:44,574 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:44,574 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:44,574 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:44,575 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:44,575 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,576 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-24 21:53:44,576 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-24 21:53:44,578 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,578 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:44,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:44,579 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,579 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:44,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:44,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:44,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:44,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-24 21:53:44,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@384e0abf[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:44,584 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-24 21:53:44,584 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,585 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,586 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-24 21:53:44,586 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-24 21:53:44,586 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:44,589 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,589 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-24 21:53:44,589 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:44,589 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-24 21:53:44,590 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-24 21:53:44,590 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:44,590 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-24 21:53:44,590 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:44,591 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:44,591 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,591 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:44,593 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,593 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:44,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,593 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:44,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:44,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:44,595 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:44,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,598 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:44,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,607 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,607 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,607 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,607 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,607 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,607 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:44,607 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:44,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:44,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:44,607 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:44,607 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,607 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:44,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-24 21:53:44,608 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:44,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:44,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:44,614 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:44,715 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:44,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:44,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:44,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:44,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,734 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:44,734 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:44,744 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-24 21:53:44,744 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-24 21:53:44,745 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,745 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,746 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,747 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,747 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:44,746 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:44,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:44,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:44,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,751 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-24 21:53:44,751 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,751 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-24 21:53:44,751 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:44,752 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-24 21:53:44,752 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:44,752 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:44,752 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:44,753 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. started... 2023-03-24 21:53:44,753 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b8b55d7b0784a827760bb4f27b0708da 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:44,842 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120522ee3d67197b4c9fb112f387e6936199 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120522ee3d67197b4c9fb112f387e6936199 2023-03-24 21:53:44,882 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/2ec2e5b8449a43bb92b59831fa0734d3 2023-03-24 21:53:44,895 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/2ec2e5b8449a43bb92b59831fa0734d3 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/2ec2e5b8449a43bb92b59831fa0734d3 2023-03-24 21:53:44,906 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/2ec2e5b8449a43bb92b59831fa0734d3, entries=1, sequenceid=17, filesize=5.0 K 2023-03-24 21:53:44,908 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b8b55d7b0784a827760bb4f27b0708da in 155ms, sequenceid=17, compaction requested=true 2023-03-24 21:53:44,908 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:44,908 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:44,908 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:44,909 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:44,909 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,909 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-24 21:53:44,909 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-24 21:53:44,914 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:44,914 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:44,915 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,915 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:44,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:44,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-24 21:53:44,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@265715f4[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:44,919 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-24 21:53:44,919 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,922 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,923 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:44,934 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,934 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-24 21:53:44,934 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-24 21:53:44,934 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:44,939 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:44,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:44,939 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-24 21:53:44,939 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:44,939 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-24 21:53:44,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:44,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:44,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:44,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:44,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-24 21:53:44,946 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-24 21:53:44,946 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:44,946 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:44,946 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,946 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:44,948 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,949 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:44,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,949 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:44,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:44,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:44,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:44,949 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:44,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:44,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,954 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:44,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,961 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:44,961 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:44,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:44,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:44,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:44,962 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:44,962 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-24 21:53:44,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:44,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:44,962 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:44,962 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,963 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,963 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:44,963 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,963 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,963 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:44,964 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:44,964 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:45,064 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:45,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:45,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:45,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:45,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:45,101 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:45,101 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:45,102 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-24 21:53:45,102 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-24 21:53:45,103 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:45,103 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:45,104 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:45,104 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:45,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:45,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:45,105 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:45,105 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:45,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:45,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:45,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:45,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:45,108 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:45,109 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-24 21:53:45,109 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:45,109 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-24 21:53:45,109 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:45,110 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-24 21:53:45,110 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:45,110 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:45,110 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:45,110 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. started... 2023-03-24 21:53:45,111 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b8b55d7b0784a827760bb4f27b0708da 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:45,586 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228bb258b61eae745d5b0c81e532b5d77eb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228bb258b61eae745d5b0c81e532b5d77eb 2023-03-24 21:53:46,009 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/612d84b41f3541cab1b2fd4f1e94c4fc 2023-03-24 21:53:46,022 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/612d84b41f3541cab1b2fd4f1e94c4fc as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/612d84b41f3541cab1b2fd4f1e94c4fc 2023-03-24 21:53:46,037 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/612d84b41f3541cab1b2fd4f1e94c4fc, entries=1, sequenceid=21, filesize=5.0 K 2023-03-24 21:53:46,038 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b8b55d7b0784a827760bb4f27b0708da in 928ms, sequenceid=21, compaction requested=true 2023-03-24 21:53:46,038 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:46,038 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:46,038 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:46,040 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:46,040 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,040 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-24 21:53:46,040 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-24 21:53:46,045 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:46,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:46,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:46,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:46,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:46,048 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-24 21:53:46,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5acaa4c4[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:46,049 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-24 21:53:46,049 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,045 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,053 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(162): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,054 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,054 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:46,054 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-24 21:53:46,054 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-24 21:53:46,054 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:46,054 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,054 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:46,056 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:46,056 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-24 21:53:46,056 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:46,056 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-24 21:53:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:46,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:46,061 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-24 21:53:46,061 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-24 21:53:46,061 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:46,061 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:46,061 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,062 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:46,064 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,064 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:46,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,064 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:46,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:46,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:46,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:46,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:46,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,066 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:46,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,069 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:46,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,074 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,074 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,074 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,074 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,074 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:46,074 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-24 21:53:46,074 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:46,075 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,075 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:46,074 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,075 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,075 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:46,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:46,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:46,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:46,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:46,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:46,081 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:46,081 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:46,181 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:46,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:46,194 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:46,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:46,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,199 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:46,199 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:46,199 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-24 21:53:46,199 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-24 21:53:46,200 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,201 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,203 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:46,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:46,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:46,204 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,204 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,204 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,204 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:46,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:46,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,205 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,205 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-24 21:53:46,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,205 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-24 21:53:46,205 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:46,206 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-24 21:53:46,206 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:46,206 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:46,206 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:46,206 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. started... 2023-03-24 21:53:46,206 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b8b55d7b0784a827760bb4f27b0708da 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:46,632 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-24 21:53:46,683 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123146140ab40d02412581a56bfd9986ecdb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123146140ab40d02412581a56bfd9986ecdb 2023-03-24 21:53:46,750 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/2d785eb4765b4e78944acf63dbc35341 2023-03-24 21:53:46,765 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/2d785eb4765b4e78944acf63dbc35341 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/2d785eb4765b4e78944acf63dbc35341 2023-03-24 21:53:46,790 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/2d785eb4765b4e78944acf63dbc35341, entries=1, sequenceid=25, filesize=5.0 K 2023-03-24 21:53:46,793 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b8b55d7b0784a827760bb4f27b0708da in 587ms, sequenceid=25, compaction requested=true 2023-03-24 21:53:46,793 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:46,793 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:46,793 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:46,794 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:46,795 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,795 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-24 21:53:46,795 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-24 21:53:46,800 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,801 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:46,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:46,801 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,801 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:46,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-24 21:53:46,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2408f086[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:46,804 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-24 21:53:46,804 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,806 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,806 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-24 21:53:46,806 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-24 21:53:46,806 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:46,807 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,807 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:46,808 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-24 21:53:46,808 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:46,809 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,809 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-24 21:53:46,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:46,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:46,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:46,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:46,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:46,830 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-03-24 21:53:46,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:46,833 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-24 21:53:46,833 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-24 21:53:46,833 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:46,833 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:46,833 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,833 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:46,835 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,835 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:46,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,835 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:46,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:46,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:46,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:46,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:46,836 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,837 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-03-24 21:53:46,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:46,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,839 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:46,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,843 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:46,843 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,843 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,843 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:46,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:46,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:46,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:46,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:46,843 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:46,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-24 21:53:46,843 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,845 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,845 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,845 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,845 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,845 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:46,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:46,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:46,845 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:46,945 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:46,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:46,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:46,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:46,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,959 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:46,959 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:46,960 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-24 21:53:46,960 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-24 21:53:46,961 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,961 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,962 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:46,962 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:46,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:46,963 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:46,963 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:46,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:46,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,964 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,967 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-24 21:53:46,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:46,967 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-24 21:53:46,967 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:46,968 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-24 21:53:46,979 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:46,979 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:46,979 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:46,979 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. started... 2023-03-24 21:53:46,979 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b8b55d7b0784a827760bb4f27b0708da 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:47,062 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010195b935aa2f5c450f82d80c7be395288d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010195b935aa2f5c450f82d80c7be395288d 2023-03-24 21:53:47,116 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/871513d4fe7946a88806f416230901ee 2023-03-24 21:53:47,126 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/871513d4fe7946a88806f416230901ee as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/871513d4fe7946a88806f416230901ee 2023-03-24 21:53:47,148 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/871513d4fe7946a88806f416230901ee, entries=1, sequenceid=29, filesize=5.0 K 2023-03-24 21:53:47,150 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b8b55d7b0784a827760bb4f27b0708da in 171ms, sequenceid=29, compaction requested=true 2023-03-24 21:53:47,151 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:47,151 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:47,151 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:47,152 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:47,152 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,152 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-24 21:53:47,152 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-24 21:53:47,154 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,154 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:47,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:47,155 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,155 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:47,164 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-24 21:53:47,164 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c5b0d4a[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:47,164 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-24 21:53:47,164 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,165 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,166 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-24 21:53:47,166 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-24 21:53:47,166 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:47,166 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,166 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:47,169 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-24 21:53:47,169 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:47,169 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-24 21:53:47,169 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:47,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:47,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:47,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:47,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:47,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:47,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-24 21:53:47,174 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-24 21:53:47,174 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:47,174 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:47,174 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,174 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:47,177 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:47,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:47,177 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,177 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:47,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:47,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:47,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:47,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,179 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:47,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,181 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:47,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,187 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:47,187 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:47,187 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:47,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:47,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:47,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:47,187 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:47,187 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,188 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,188 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,188 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,189 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,189 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,189 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:47,190 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:47,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:47,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:47,290 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:47,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:47,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:47,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:47,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,311 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:47,311 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:47,312 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-24 21:53:47,312 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-24 21:53:47,312 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,312 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,315 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:47,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:47,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:47,316 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,317 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,317 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,317 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:47,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:47,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,318 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,318 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-24 21:53:47,318 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,318 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-24 21:53:47,318 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:47,319 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-24 21:53:47,319 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:47,319 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:47,319 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:47,319 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. started... 2023-03-24 21:53:47,319 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b8b55d7b0784a827760bb4f27b0708da 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:47,376 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010387d6503865ee408da59f7ec12515dcec to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010387d6503865ee408da59f7ec12515dcec 2023-03-24 21:53:47,810 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/8325f193518941bb8cc62e8eed83fa61 2023-03-24 21:53:47,822 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/8325f193518941bb8cc62e8eed83fa61 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/8325f193518941bb8cc62e8eed83fa61 2023-03-24 21:53:47,911 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/8325f193518941bb8cc62e8eed83fa61, entries=1, sequenceid=33, filesize=5.0 K 2023-03-24 21:53:47,913 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b8b55d7b0784a827760bb4f27b0708da in 594ms, sequenceid=33, compaction requested=true 2023-03-24 21:53:47,913 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:47,914 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:47,914 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:47,914 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:47,914 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,914 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-24 21:53:47,914 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-24 21:53:47,917 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,917 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:47,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:47,917 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,917 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:47,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:47,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:47,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:47,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-24 21:53:47,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@723d5d80[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:47,920 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-24 21:53:47,921 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,922 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,922 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-24 21:53:47,922 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-24 21:53:47,922 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,923 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:47,923 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:47,926 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:47,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:47,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:47,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:47,927 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-24 21:53:47,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,927 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:47,927 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-24 21:53:47,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:47,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:47,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-24 21:53:47,948 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-24 21:53:47,948 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:47,948 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:47,948 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,948 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:47,951 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,951 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,951 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:47,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:47,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:47,951 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:47,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:47,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:47,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:47,954 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:47,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,958 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:47,958 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:47,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:47,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:47,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:47,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:47,962 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,962 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,962 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,962 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:47,962 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:47,962 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:47,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:47,963 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,963 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:47,964 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:47,964 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:47,964 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-24 21:53:47,997 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-24 21:53:47,997 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-03-24 21:53:48,064 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:48,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:48,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:48,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:48,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,090 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:48,100 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:48,102 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-24 21:53:48,102 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-24 21:53:48,103 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,103 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,105 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:48,105 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:48,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:48,105 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,105 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:48,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:48,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,107 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,107 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-24 21:53:48,108 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,108 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-24 21:53:48,108 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:48,108 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-24 21:53:48,109 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:48,109 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:48,110 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:48,110 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. started... 2023-03-24 21:53:48,110 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b8b55d7b0784a827760bb4f27b0708da 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:48,208 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907d5d0296b28d34bda830cc4a27db3e6f2 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20150907d5d0296b28d34bda830cc4a27db3e6f2 2023-03-24 21:53:48,627 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/ee89172fa6a84ca9b120e2b404fb76cb 2023-03-24 21:53:48,641 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/ee89172fa6a84ca9b120e2b404fb76cb as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ee89172fa6a84ca9b120e2b404fb76cb 2023-03-24 21:53:48,650 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ee89172fa6a84ca9b120e2b404fb76cb, entries=1, sequenceid=37, filesize=5.0 K 2023-03-24 21:53:48,651 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b8b55d7b0784a827760bb4f27b0708da in 541ms, sequenceid=37, compaction requested=true 2023-03-24 21:53:48,651 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:48,651 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:48,651 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:48,652 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:48,652 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,652 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-24 21:53:48,652 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-24 21:53:48,654 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,654 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:48,654 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,654 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:48,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:48,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:48,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-24 21:53:48,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d5e3b87[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:48,656 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-24 21:53:48,657 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,658 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,658 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-24 21:53:48,658 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-24 21:53:48,658 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:48,658 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,659 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:48,662 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,662 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-24 21:53:48,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,662 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:48,662 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-24 21:53:48,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:48,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:48,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:48,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:48,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:48,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:48,669 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-24 21:53:48,669 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-24 21:53:48,669 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:48,669 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:48,669 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,669 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:48,672 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:48,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:48,673 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,673 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:48,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:48,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:48,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:48,674 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:48,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,676 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:48,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,680 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,680 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,680 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,680 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,680 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,680 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:48,680 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:48,680 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:48,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:48,680 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:48,680 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:48,680 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,681 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,681 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-24 21:53:48,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:48,681 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:48,681 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:48,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:48,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:48,782 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:48,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:48,793 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:48,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:48,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,796 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:48,796 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:48,797 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-24 21:53:48,797 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-24 21:53:48,797 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,797 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,799 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:48,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:48,799 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:48,800 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,800 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:48,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,801 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:48,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,802 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,808 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-24 21:53:48,808 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,808 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-24 21:53:48,809 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:48,810 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-24 21:53:48,810 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:48,810 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:48,810 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:48,811 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. started... 2023-03-24 21:53:48,811 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b8b55d7b0784a827760bb4f27b0708da 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:48,929 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120b6e0cd1cd0c047b892ff1bf5e2202515 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120b6e0cd1cd0c047b892ff1bf5e2202515 2023-03-24 21:53:48,951 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/20446636ae11466abc79286f5fc69049 2023-03-24 21:53:48,963 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/20446636ae11466abc79286f5fc69049 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/20446636ae11466abc79286f5fc69049 2023-03-24 21:53:48,974 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/20446636ae11466abc79286f5fc69049, entries=1, sequenceid=41, filesize=5.0 K 2023-03-24 21:53:48,976 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b8b55d7b0784a827760bb4f27b0708da in 165ms, sequenceid=41, compaction requested=true 2023-03-24 21:53:48,976 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:48,976 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:48,977 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:48,977 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:48,977 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,977 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-24 21:53:48,978 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-24 21:53:48,980 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,980 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:48,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:48,980 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:48,980 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:48,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:48,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:48,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-24 21:53:48,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1e31686a[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:48,983 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-24 21:53:48,983 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,985 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,985 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-24 21:53:48,985 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-24 21:53:48,986 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:48,986 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,986 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:48,990 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-24 21:53:48,990 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:48,990 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-24 21:53:48,990 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:48,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:48,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:48,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:48,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:48,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:48,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:48,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:49,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:49,002 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-24 21:53:49,002 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-24 21:53:49,002 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:49,002 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:49,002 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,002 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:49,006 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,006 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:49,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,007 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:49,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:49,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:49,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:49,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:49,011 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:49,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:49,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:49,016 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:49,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:49,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:49,023 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:49,023 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,023 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,023 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:49,023 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:49,023 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,023 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:49,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:49,023 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:49,024 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:49,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:49,026 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:49,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-24 21:53:49,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:49,026 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:49,027 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:49,127 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:49,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:49,135 DEBUG [Listener at localhost/39907] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-24 21:53:49,140 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:48250, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-24 21:53:49,143 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:49,178 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-24 21:53:49,179 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-24 21:53:49,179 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:53:49,181 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695429181 2023-03-24 21:53:49,181 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-24 21:53:49,186 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-24 21:53:49,186 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-24 21:53:49,187 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x5ed3108e to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:53:49,192 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e84ab8c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:53:49,194 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:49,421 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c4dafa095c4347119885f52d0e631279 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103c4dafa095c4347119885f52d0e631279 2023-03-24 21:53:49,423 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-24 21:53:49,465 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:53:49,467 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:52764, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:53:49,503 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:53:49,507 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:48260, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:53:49,593 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f35df6f9fddc492a93e50dfa84b6406b first=Optional[r3] last=Optional[r6] 2023-03-24 21:53:49,624 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f35df6f9fddc492a93e50dfa84b6406b}] 2023-03-24 21:53:49,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f35df6f9fddc492a93e50dfa84b6406b for inclusion in b8b55d7b0784a827760bb4f27b0708da/family1 2023-03-24 21:53:49,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-24 21:53:49,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:53:49,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:49,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f35df6f9fddc492a93e50dfa84b6406b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gfhufersj69hct62k5uuqm639u3g8tpndpq4ml8lkc87m0fktmb8kqr10nco8565/family1/f35df6f9fddc492a93e50dfa84b6406b 2023-03-24 21:53:49,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gfhufersj69hct62k5uuqm639u3g8tpndpq4ml8lkc87m0fktmb8kqr10nco8565/family1/f35df6f9fddc492a93e50dfa84b6406b as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/a6696c58f34a4a43a3bdb869b1b40cf6_SeqId_44_ 2023-03-24 21:53:49,665 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gfhufersj69hct62k5uuqm639u3g8tpndpq4ml8lkc87m0fktmb8kqr10nco8565/family1/f35df6f9fddc492a93e50dfa84b6406b into b8b55d7b0784a827760bb4f27b0708da/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/a6696c58f34a4a43a3bdb869b1b40cf6_SeqId_44_ - updating store file list. 2023-03-24 21:53:49,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/a6696c58f34a4a43a3bdb869b1b40cf6_SeqId_44_ into b8b55d7b0784a827760bb4f27b0708da/family1 2023-03-24 21:53:49,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gfhufersj69hct62k5uuqm639u3g8tpndpq4ml8lkc87m0fktmb8kqr10nco8565/family1/f35df6f9fddc492a93e50dfa84b6406b into b8b55d7b0784a827760bb4f27b0708da/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/a6696c58f34a4a43a3bdb869b1b40cf6_SeqId_44_) 2023-03-24 21:53:49,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gfhufersj69hct62k5uuqm639u3g8tpndpq4ml8lkc87m0fktmb8kqr10nco8565/family1/f35df6f9fddc492a93e50dfa84b6406b 2023-03-24 21:53:49,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-24 21:53:49,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-24 21:53:49,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(1912): b8b55d7b0784a827760bb4f27b0708da/family1 is initiating minor compaction 2023-03-24 21:53:49,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4b5b18f3; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:53:49,706 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b8b55d7b0784a827760bb4f27b0708da/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:49,711 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/5d589de0e0094f18a8a6a6f9de26661a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/577435a1966f43449ea06e6a5f3e10f1, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/2ec2e5b8449a43bb92b59831fa0734d3, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/612d84b41f3541cab1b2fd4f1e94c4fc, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/2d785eb4765b4e78944acf63dbc35341, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/871513d4fe7946a88806f416230901ee, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/8325f193518941bb8cc62e8eed83fa61, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ee89172fa6a84ca9b120e2b404fb76cb, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/20446636ae11466abc79286f5fc69049, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/a6696c58f34a4a43a3bdb869b1b40cf6_SeqId_44_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp, totalSize=50.2 K 2023-03-24 21:53:49,712 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 5d589de0e0094f18a8a6a6f9de26661a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-24 21:53:49,712 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 577435a1966f43449ea06e6a5f3e10f1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-24 21:53:49,712 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 2ec2e5b8449a43bb92b59831fa0734d3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-24 21:53:49,713 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 612d84b41f3541cab1b2fd4f1e94c4fc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-24 21:53:49,713 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 2d785eb4765b4e78944acf63dbc35341, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-24 21:53:49,713 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 871513d4fe7946a88806f416230901ee, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-24 21:53:49,713 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 8325f193518941bb8cc62e8eed83fa61, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-24 21:53:49,713 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting ee89172fa6a84ca9b120e2b404fb76cb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-24 21:53:49,713 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 20446636ae11466abc79286f5fc69049, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-24 21:53:49,713 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting a6696c58f34a4a43a3bdb869b1b40cf6_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-24 21:53:49,726 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-24 21:53:49,729 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:53:49,748 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228bb258b61eae745d5b0c81e532b5d77eb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228bb258b61eae745d5b0c81e532b5d77eb 2023-03-24 21:53:49,751 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123146140ab40d02412581a56bfd9986ecdb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123146140ab40d02412581a56bfd9986ecdb 2023-03-24 21:53:49,760 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010195b935aa2f5c450f82d80c7be395288d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010195b935aa2f5c450f82d80c7be395288d 2023-03-24 21:53:49,764 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010387d6503865ee408da59f7ec12515dcec to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010387d6503865ee408da59f7ec12515dcec 2023-03-24 21:53:49,764 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-03-24 21:53:49,765 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-24 21:53:49,766 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:49,766 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:49,766 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:49,766 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:49,767 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:49,767 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:49,767 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:49,767 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:49,869 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): b8b55d7b0784a827760bb4f27b0708da#family1#compaction#10 average throughput is 0.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:53:50,405 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/85cbd548c31f43be86d493f332a87e1b as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/85cbd548c31f43be86d493f332a87e1b 2023-03-24 21:53:50,438 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in b8b55d7b0784a827760bb4f27b0708da/family1 of b8b55d7b0784a827760bb4f27b0708da into 85cbd548c31f43be86d493f332a87e1b(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:53:50,438 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:50,440 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1679694829699; duration=0sec 2023-03-24 21:53:50,441 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:53:50,467 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512059491a3ee8e71456cb4abca5bc55c141b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512059491a3ee8e71456cb4abca5bc55c141b 2023-03-24 21:53:50,483 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-24 21:53:50,505 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/5d589de0e0094f18a8a6a6f9de26661a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/577435a1966f43449ea06e6a5f3e10f1, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/2ec2e5b8449a43bb92b59831fa0734d3, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/612d84b41f3541cab1b2fd4f1e94c4fc, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/2d785eb4765b4e78944acf63dbc35341, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/871513d4fe7946a88806f416230901ee, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/8325f193518941bb8cc62e8eed83fa61, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ee89172fa6a84ca9b120e2b404fb76cb, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/20446636ae11466abc79286f5fc69049, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/a6696c58f34a4a43a3bdb869b1b40cf6_SeqId_44_] to archive 2023-03-24 21:53:50,507 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:53:50,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/5d589de0e0094f18a8a6a6f9de26661a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/5d589de0e0094f18a8a6a6f9de26661a 2023-03-24 21:53:50,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/577435a1966f43449ea06e6a5f3e10f1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/577435a1966f43449ea06e6a5f3e10f1 2023-03-24 21:53:50,518 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/2ec2e5b8449a43bb92b59831fa0734d3 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/2ec2e5b8449a43bb92b59831fa0734d3 2023-03-24 21:53:50,522 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/612d84b41f3541cab1b2fd4f1e94c4fc to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/612d84b41f3541cab1b2fd4f1e94c4fc 2023-03-24 21:53:50,525 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/2d785eb4765b4e78944acf63dbc35341 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/2d785eb4765b4e78944acf63dbc35341 2023-03-24 21:53:50,527 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/871513d4fe7946a88806f416230901ee to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/871513d4fe7946a88806f416230901ee 2023-03-24 21:53:50,532 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/8325f193518941bb8cc62e8eed83fa61 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/8325f193518941bb8cc62e8eed83fa61 2023-03-24 21:53:50,534 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ee89172fa6a84ca9b120e2b404fb76cb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ee89172fa6a84ca9b120e2b404fb76cb 2023-03-24 21:53:50,536 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/20446636ae11466abc79286f5fc69049 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/20446636ae11466abc79286f5fc69049 2023-03-24 21:53:50,538 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/a6696c58f34a4a43a3bdb869b1b40cf6_SeqId_44_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/a6696c58f34a4a43a3bdb869b1b40cf6_SeqId_44_ 2023-03-24 21:53:50,550 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b3470231cd0144aa8349f279271af1eb first=Optional[r00] last=Optional[r2] 2023-03-24 21:53:50,552 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b3470231cd0144aa8349f279271af1eb}] 2023-03-24 21:53:50,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b3470231cd0144aa8349f279271af1eb for inclusion in b8b55d7b0784a827760bb4f27b0708da/family1 2023-03-24 21:53:50,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-24 21:53:50,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:53:50,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:50,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b3470231cd0144aa8349f279271af1eb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__h20krmrepph9rar10vic72e40eohc02rpg9158gemvvk6m50krsropa4luh7rnd8/family1/b3470231cd0144aa8349f279271af1eb 2023-03-24 21:53:50,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__h20krmrepph9rar10vic72e40eohc02rpg9158gemvvk6m50krsropa4luh7rnd8/family1/b3470231cd0144aa8349f279271af1eb as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ba51a409a0fe4324abe786efd18d6527_SeqId_47_ 2023-03-24 21:53:50,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__h20krmrepph9rar10vic72e40eohc02rpg9158gemvvk6m50krsropa4luh7rnd8/family1/b3470231cd0144aa8349f279271af1eb into b8b55d7b0784a827760bb4f27b0708da/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ba51a409a0fe4324abe786efd18d6527_SeqId_47_ - updating store file list. 2023-03-24 21:53:50,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ba51a409a0fe4324abe786efd18d6527_SeqId_47_ into b8b55d7b0784a827760bb4f27b0708da/family1 2023-03-24 21:53:50,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__h20krmrepph9rar10vic72e40eohc02rpg9158gemvvk6m50krsropa4luh7rnd8/family1/b3470231cd0144aa8349f279271af1eb into b8b55d7b0784a827760bb4f27b0708da/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ba51a409a0fe4324abe786efd18d6527_SeqId_47_) 2023-03-24 21:53:50,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__h20krmrepph9rar10vic72e40eohc02rpg9158gemvvk6m50krsropa4luh7rnd8/family1/b3470231cd0144aa8349f279271af1eb 2023-03-24 21:53:50,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-24 21:53:50,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-24 21:53:50,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(1912): b8b55d7b0784a827760bb4f27b0708da/family1 is initiating minor compaction (all files) 2023-03-24 21:53:50,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6f8c456a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:53:50,587 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b8b55d7b0784a827760bb4f27b0708da/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:50,593 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/17e04a222db94f9fa7f4e1c649c9ab4d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/85cbd548c31f43be86d493f332a87e1b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ba51a409a0fe4324abe786efd18d6527_SeqId_47_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp, totalSize=358.6 K 2023-03-24 21:53:50,594 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 17e04a222db94f9fa7f4e1c649c9ab4d, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841999 2023-03-24 21:53:50,594 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 85cbd548c31f43be86d493f332a87e1b, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241999 2023-03-24 21:53:50,595 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting ba51a409a0fe4324abe786efd18d6527_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878841999 2023-03-24 21:53:50,599 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-24 21:53:50,602 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:53:50,607 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130d45b5df239b34e5bbbca4ef00934c63f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130d45b5df239b34e5bbbca4ef00934c63f 2023-03-24 21:53:50,610 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201115b300fa5fa4c37a98127a95ffb54de to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201115b300fa5fa4c37a98127a95ffb54de 2023-03-24 21:53:50,615 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120522ee3d67197b4c9fb112f387e6936199 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120522ee3d67197b4c9fb112f387e6936199 2023-03-24 21:53:50,615 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-03-24 21:53:50,615 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-24 21:53:50,616 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:50,616 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-24 21:53:50,617 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:53:50,618 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-24 21:53:50,621 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.4600 sec 2023-03-24 21:53:50,686 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): b8b55d7b0784a827760bb4f27b0708da#family1#compaction#11 average throughput is 2.54 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:53:50,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$12(2454): Client=jenkins//172.31.2.10 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-24 21:53:50,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:51,421 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-24 21:53:51,424 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-24 21:53:51,424 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-24 21:53:51,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-03-24 21:53:51,448 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=b8b55d7b0784a827760bb4f27b0708da, REOPEN/MOVE}] 2023-03-24 21:53:51,449 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=b8b55d7b0784a827760bb4f27b0708da, REOPEN/MOVE 2023-03-24 21:53:51,450 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=b8b55d7b0784a827760bb4f27b0708da, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:51,451 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679694831450"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694831450"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694831450"}]},"ts":"1679694831450"} 2023-03-24 21:53:51,454 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure b8b55d7b0784a827760bb4f27b0708da, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:53:51,604 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/.tmp/family1/9ff69003c10d405ea03d984a84c118e2 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/9ff69003c10d405ea03d984a84c118e2 2023-03-24 21:53:51,615 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:51,616 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing b8b55d7b0784a827760bb4f27b0708da, disabling compactions & flushes 2023-03-24 21:53:51,616 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:51,620 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in b8b55d7b0784a827760bb4f27b0708da/family1 of b8b55d7b0784a827760bb4f27b0708da into 9ff69003c10d405ea03d984a84c118e2(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-24 21:53:51,622 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:51,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:51,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:51,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. after waiting 0 ms 2023-03-24 21:53:51,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:51,622 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1679694830587; duration=1sec 2023-03-24 21:53:51,622 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:53:51,626 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/17e04a222db94f9fa7f4e1c649c9ab4d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/85cbd548c31f43be86d493f332a87e1b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ba51a409a0fe4324abe786efd18d6527_SeqId_47_] to archive 2023-03-24 21:53:51,628 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:53:51,632 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/17e04a222db94f9fa7f4e1c649c9ab4d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/17e04a222db94f9fa7f4e1c649c9ab4d 2023-03-24 21:53:51,634 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/85cbd548c31f43be86d493f332a87e1b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/85cbd548c31f43be86d493f332a87e1b 2023-03-24 21:53:51,640 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ba51a409a0fe4324abe786efd18d6527_SeqId_47_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/ba51a409a0fe4324abe786efd18d6527_SeqId_47_ 2023-03-24 21:53:51,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-24 21:53:51,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:51,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:51,652 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(3509): Not adding moved region record: b8b55d7b0784a827760bb4f27b0708da to self. 2023-03-24 21:53:51,656 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:51,656 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=b8b55d7b0784a827760bb4f27b0708da, regionState=CLOSED 2023-03-24 21:53:51,657 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679694831656"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694831656"}]},"ts":"1679694831656"} 2023-03-24 21:53:51,664 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-03-24 21:53:51,664 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure b8b55d7b0784a827760bb4f27b0708da, server=jenkins-hbase9.apache.org,41807,1679694819103 in 206 msec 2023-03-24 21:53:51,665 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=b8b55d7b0784a827760bb4f27b0708da, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=true 2023-03-24 21:53:51,816 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=b8b55d7b0784a827760bb4f27b0708da, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:51,816 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679694831816"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694831816"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694831816"}]},"ts":"1679694831816"} 2023-03-24 21:53:51,820 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure b8b55d7b0784a827760bb4f27b0708da, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:53:51,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:51,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b8b55d7b0784a827760bb4f27b0708da, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:53:51,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:51,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:53:51,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:51,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:51,981 INFO [StoreOpener-b8b55d7b0784a827760bb4f27b0708da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:51,987 DEBUG [StoreOpener-b8b55d7b0784a827760bb4f27b0708da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1 2023-03-24 21:53:51,987 DEBUG [StoreOpener-b8b55d7b0784a827760bb4f27b0708da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1 2023-03-24 21:53:51,988 INFO [StoreOpener-b8b55d7b0784a827760bb4f27b0708da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b8b55d7b0784a827760bb4f27b0708da columnFamilyName family1 2023-03-24 21:53:52,004 DEBUG [StoreOpener-b8b55d7b0784a827760bb4f27b0708da-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/9ff69003c10d405ea03d984a84c118e2 2023-03-24 21:53:52,005 INFO [StoreOpener-b8b55d7b0784a827760bb4f27b0708da-1] regionserver.HStore(310): Store=b8b55d7b0784a827760bb4f27b0708da/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:53:52,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:52,008 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:52,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:53:52,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened b8b55d7b0784a827760bb4f27b0708da; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60969077, jitterRate=-0.09148995578289032}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:53:52,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:52,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da., pid=17, masterSystemTime=1679694831973 2023-03-24 21:53:52,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:52,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:53:52,018 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=b8b55d7b0784a827760bb4f27b0708da, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:52,018 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679694832018"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694832018"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694832018"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694832018"}]},"ts":"1679694832018"} 2023-03-24 21:53:52,023 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-03-24 21:53:52,023 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure b8b55d7b0784a827760bb4f27b0708da, server=jenkins-hbase9.apache.org,41807,1679694819103 in 200 msec 2023-03-24 21:53:52,025 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-03-24 21:53:52,026 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=b8b55d7b0784a827760bb4f27b0708da, REOPEN/MOVE in 575 msec 2023-03-24 21:53:52,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-03-24 21:53:52,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 599 msec 2023-03-24 21:53:52,033 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0800 sec 2023-03-24 21:53:52,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-03-24 21:53:52,045 INFO [Listener at localhost/39907] compactions.TestMobCompactor(278): alter status finished 2023-03-24 21:53:52,047 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-24 21:53:52,049 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-24 21:53:52,051 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-24 21:53:52,053 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695432053 2023-03-24 21:53:52,054 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:53:52,055 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-24 21:53:52,056 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-24 21:53:52,056 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-24 21:53:52,056 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x2d3f3388 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:53:52,068 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4f3b3c97, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:53:52,068 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:52,068 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:52,069 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:52,069 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:52,069 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:52,176 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128f5f679b68086457ca6e5d172e8dad736 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128f5f679b68086457ca6e5d172e8dad736 2023-03-24 21:53:52,177 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-24 21:53:52,194 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:53:52,195 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:49428, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:53:52,209 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:53:52,211 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:46788, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:53:52,254 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b3e6d5e71d154957990757dbb2d34d61 first=Optional[r05] last=Optional[r8] 2023-03-24 21:53:52,255 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=51 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b3e6d5e71d154957990757dbb2d34d61}] 2023-03-24 21:53:52,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b3e6d5e71d154957990757dbb2d34d61 for inclusion in b8b55d7b0784a827760bb4f27b0708da/family1 2023-03-24 21:53:52,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-24 21:53:52,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:53:52,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:53:52,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b3e6d5e71d154957990757dbb2d34d61 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__23ju7u3ouaf1ralnjmt02q8bvhg71dllvrsqfm04apv9r2076j6ktcq08inou92v/family1/b3e6d5e71d154957990757dbb2d34d61 2023-03-24 21:53:52,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__23ju7u3ouaf1ralnjmt02q8bvhg71dllvrsqfm04apv9r2076j6ktcq08inou92v/family1/b3e6d5e71d154957990757dbb2d34d61 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/c15ee96b3a884cc2bf490a5e7df6c424_SeqId_53_ 2023-03-24 21:53:52,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__23ju7u3ouaf1ralnjmt02q8bvhg71dllvrsqfm04apv9r2076j6ktcq08inou92v/family1/b3e6d5e71d154957990757dbb2d34d61 into b8b55d7b0784a827760bb4f27b0708da/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/c15ee96b3a884cc2bf490a5e7df6c424_SeqId_53_ - updating store file list. 2023-03-24 21:53:52,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/c15ee96b3a884cc2bf490a5e7df6c424_SeqId_53_ into b8b55d7b0784a827760bb4f27b0708da/family1 2023-03-24 21:53:52,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__23ju7u3ouaf1ralnjmt02q8bvhg71dllvrsqfm04apv9r2076j6ktcq08inou92v/family1/b3e6d5e71d154957990757dbb2d34d61 into b8b55d7b0784a827760bb4f27b0708da/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/family1/c15ee96b3a884cc2bf490a5e7df6c424_SeqId_53_) 2023-03-24 21:53:52,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__23ju7u3ouaf1ralnjmt02q8bvhg71dllvrsqfm04apv9r2076j6ktcq08inou92v/family1/b3e6d5e71d154957990757dbb2d34d61 2023-03-24 21:53:52,292 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-24 21:53:52,294 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:53:52,296 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120b6e0cd1cd0c047b892ff1bf5e2202515 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120b6e0cd1cd0c047b892ff1bf5e2202515 2023-03-24 21:53:52,300 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511288e9529b271f14d7fa8684e6ec8f3579a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511288e9529b271f14d7fa8684e6ec8f3579a 2023-03-24 21:53:52,300 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-03-24 21:53:52,300 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-24 21:53:52,300 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:52,301 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:52,301 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-24 21:53:52,301 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-24 21:53:52,301 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:53:52,303 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-24 21:53:52,306 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 255 msec 2023-03-24 21:53:52,431 INFO [Listener at localhost/39907] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=254 (was 211) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55192 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55162 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55120 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55388 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3abe1f4d-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_869545651_15 at /127.0.0.1:55346 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55352 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55428 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54680@0x2d3f3388 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/168963814.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3abe1f4d-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase9:41807-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54680@0x2d3f3388-SendThread(127.0.0.1:54680) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/cluster_90ef77c7-dcec-9426-4d35-a9af4e7f4c97/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55308 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2810e976-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55282 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54680@0x5ed3108e-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55298 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55262 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2810e976-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3abe1f4d-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:51710 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3abe1f4d-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55414 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54680@0x5ed3108e sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/168963814.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x2810e976-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55224 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55374 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54680@0x2d3f3388-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55278 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54680@0x5ed3108e-SendThread(127.0.0.1:54680) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55314 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3abe1f4d-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=601 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=377 (was 291) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 168) - ProcessCount LEAK? -, AvailableMemoryMB=8538 (was 8991) 2023-03-24 21:53:52,455 INFO [Listener at localhost/39907] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=254, OpenFileDescriptor=601, MaxFileDescriptor=60000, SystemLoadAverage=377, ProcessCount=170, AvailableMemoryMB=8538 2023-03-24 21:53:52,458 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-24 21:53:52,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:52,463 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:53:52,464 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:53:52,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-03-24 21:53:52,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-24 21:53:52,473 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30 2023-03-24 21:53:52,474 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30 empty. 2023-03-24 21:53:52,474 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30 2023-03-24 21:53:52,475 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-03-24 21:53:52,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-24 21:53:52,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-24 21:53:52,914 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-24 21:53:52,915 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 46db6a74ea50377062351d8426481e30, NAME => 'testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:53:52,939 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:53:52,940 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 46db6a74ea50377062351d8426481e30, disabling compactions & flushes 2023-03-24 21:53:52,940 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:52,940 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:52,940 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. after waiting 0 ms 2023-03-24 21:53:52,940 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:52,940 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:52,940 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:52,943 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:53:52,945 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679694832944"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694832944"}]},"ts":"1679694832944"} 2023-03-24 21:53:52,947 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-24 21:53:52,948 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:53:52,949 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694832948"}]},"ts":"1679694832948"} 2023-03-24 21:53:52,950 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-24 21:53:52,955 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=46db6a74ea50377062351d8426481e30, ASSIGN}] 2023-03-24 21:53:52,956 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=46db6a74ea50377062351d8426481e30, ASSIGN 2023-03-24 21:53:52,957 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=46db6a74ea50377062351d8426481e30, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:53:53,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-24 21:53:53,108 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=46db6a74ea50377062351d8426481e30, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:53,109 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679694833108"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694833108"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694833108"}]},"ts":"1679694833108"} 2023-03-24 21:53:53,113 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 46db6a74ea50377062351d8426481e30, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:53:53,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:53,270 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 46db6a74ea50377062351d8426481e30, NAME => 'testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:53:53,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 46db6a74ea50377062351d8426481e30 2023-03-24 21:53:53,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:53:53,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 46db6a74ea50377062351d8426481e30 2023-03-24 21:53:53,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 46db6a74ea50377062351d8426481e30 2023-03-24 21:53:53,273 INFO [StoreOpener-46db6a74ea50377062351d8426481e30-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 46db6a74ea50377062351d8426481e30 2023-03-24 21:53:53,274 DEBUG [StoreOpener-46db6a74ea50377062351d8426481e30-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1 2023-03-24 21:53:53,274 DEBUG [StoreOpener-46db6a74ea50377062351d8426481e30-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1 2023-03-24 21:53:53,275 INFO [StoreOpener-46db6a74ea50377062351d8426481e30-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 46db6a74ea50377062351d8426481e30 columnFamilyName family1 2023-03-24 21:53:53,280 INFO [StoreOpener-46db6a74ea50377062351d8426481e30-1] regionserver.HStore(310): Store=46db6a74ea50377062351d8426481e30/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:53:53,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30 2023-03-24 21:53:53,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30 2023-03-24 21:53:53,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 46db6a74ea50377062351d8426481e30 2023-03-24 21:53:53,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:53:53,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 46db6a74ea50377062351d8426481e30; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74557637, jitterRate=0.11099536716938019}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:53:53,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:53,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30., pid=21, masterSystemTime=1679694833265 2023-03-24 21:53:53,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:53,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:53,294 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=46db6a74ea50377062351d8426481e30, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:53,294 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679694833294"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694833294"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694833294"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694833294"}]},"ts":"1679694833294"} 2023-03-24 21:53:53,301 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-03-24 21:53:53,301 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 46db6a74ea50377062351d8426481e30, server=jenkins-hbase9.apache.org,41807,1679694819103 in 183 msec 2023-03-24 21:53:53,304 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-03-24 21:53:53,304 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=46db6a74ea50377062351d8426481e30, ASSIGN in 346 msec 2023-03-24 21:53:53,305 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:53:53,306 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694833305"}]},"ts":"1679694833305"} 2023-03-24 21:53:53,307 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-24 21:53:53,310 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:53:53,312 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 851 msec 2023-03-24 21:53:53,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-24 21:53:53,585 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-03-24 21:53:53,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:53,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:53,647 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:53,647 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:53,648 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:53:53,648 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:53:53,648 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:53,649 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:53,650 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:53,650 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:53,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:53,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:53,650 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:53,651 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:53,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:53,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:53,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:53,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:53,652 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:53,653 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:53,654 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:53:53,654 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:53,655 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:53:53,655 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:53,655 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:53,659 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:53,659 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. started... 2023-03-24 21:53:53,660 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 46db6a74ea50377062351d8426481e30 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-24 21:53:53,722 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511306c710908e0a245249dc07a66b7a6981e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511306c710908e0a245249dc07a66b7a6981e 2023-03-24 21:53:54,128 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/20e26bd69914450ba8d5d9e247730c01 2023-03-24 21:53:54,136 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/20e26bd69914450ba8d5d9e247730c01 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/20e26bd69914450ba8d5d9e247730c01 2023-03-24 21:53:54,144 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/20e26bd69914450ba8d5d9e247730c01, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-24 21:53:54,146 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 46db6a74ea50377062351d8426481e30 in 485ms, sequenceid=5, compaction requested=false 2023-03-24 21:53:54,146 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:54,147 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:54,148 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:54,148 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:54,148 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:54,148 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,148 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:53:54,148 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:53:54,150 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,150 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:54,150 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,150 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:54,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:54,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:54,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:53:54,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a3407bb[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:54,153 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:53:54,153 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,154 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,155 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:53:54,155 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:53:54,155 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:54,155 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,155 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:54,158 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:53:54,158 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,158 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:54,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:54,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:54,158 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:53:54,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:54,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:54,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:54,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:54,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:53:54,163 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:53:54,163 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:54,163 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:54,163 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,163 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:54,167 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,167 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:54,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:54,167 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:54,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:54,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:54,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:54,168 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:54,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,169 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:54,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:54,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:54,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:54,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:54,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:53:54,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:54,174 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:54,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:54,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:54,173 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:54,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,174 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,175 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,175 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,175 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:54,175 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:54,175 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,175 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,275 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:54,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:54,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:54,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:54,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,285 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:54,286 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:54,286 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:53:54,286 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:53:54,287 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,287 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,288 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:54,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:54,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:54,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,289 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:54,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,290 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,290 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:53:54,290 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,290 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:53:54,290 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:54,291 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:53:54,291 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,291 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,291 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:54,293 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:54,293 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:54,293 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:54,293 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. started... 2023-03-24 21:53:54,293 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 46db6a74ea50377062351d8426481e30 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-24 21:53:54,731 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128295679f4a15a4915aa57297a4b4a061a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128295679f4a15a4915aa57297a4b4a061a 2023-03-24 21:53:54,744 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/2b2de07382b54595897ea641e98c5e01 2023-03-24 21:53:54,751 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/2b2de07382b54595897ea641e98c5e01 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/2b2de07382b54595897ea641e98c5e01 2023-03-24 21:53:54,759 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/2b2de07382b54595897ea641e98c5e01, entries=1, sequenceid=9, filesize=5.0 K 2023-03-24 21:53:54,760 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 46db6a74ea50377062351d8426481e30 in 467ms, sequenceid=9, compaction requested=false 2023-03-24 21:53:54,762 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:54,763 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:54,763 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:54,763 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:54,763 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,763 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:53:54,763 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:53:54,765 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,765 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:54,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:54,766 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,766 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:54,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:54,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:54,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:54,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:53:54,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@327d8e20[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:54,768 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:53:54,768 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,772 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,772 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:53:54,773 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:53:54,773 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:54,773 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,773 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:54,775 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:53:54,775 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:54,775 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:53:54,775 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:54,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:54,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:53:54,779 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:53:54,779 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:54,779 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:54,779 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,779 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:54,782 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,783 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,784 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,785 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,785 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:54,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:54,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:54,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:54,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:54,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:54,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:54,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,804 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:54,804 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:54,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:54,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:54,804 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:54,804 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,804 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:54,804 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,804 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,804 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,804 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,804 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:54,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:54,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:54,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:54,807 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:54,807 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:54,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:53:54,907 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:54,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:54,916 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:54,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:54,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,926 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:54,926 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:54,926 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:53:54,926 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:53:54,927 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,927 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,929 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,929 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:54,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:54,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:54,930 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,930 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:54,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,930 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:54,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,931 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,931 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:53:54,931 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,931 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:53:54,931 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:54,932 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:53:54,935 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:54,935 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:54,935 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:54,935 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. started... 2023-03-24 21:53:54,935 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 46db6a74ea50377062351d8426481e30 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:54,954 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512010fc16511e4dd4598b50900b93d9e6097 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512010fc16511e4dd4598b50900b93d9e6097 2023-03-24 21:53:54,965 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/094ad5929e5f4c048ea457912fd5029d 2023-03-24 21:53:54,972 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/094ad5929e5f4c048ea457912fd5029d as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/094ad5929e5f4c048ea457912fd5029d 2023-03-24 21:53:54,980 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/094ad5929e5f4c048ea457912fd5029d, entries=1, sequenceid=13, filesize=4.9 K 2023-03-24 21:53:54,981 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 46db6a74ea50377062351d8426481e30 in 46ms, sequenceid=13, compaction requested=true 2023-03-24 21:53:54,982 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:54,982 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:54,982 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:54,982 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:54,982 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,982 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:53:54,982 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:53:54,984 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,984 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:54,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:54,985 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,985 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:54,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:54,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:54,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:54,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:53:54,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ef319eb[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:54,987 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:53:54,987 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,989 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,989 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:53:54,989 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:53:54,989 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,989 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:54,989 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:54,992 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:53:54,992 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,992 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:54,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:54,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:54,993 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:53:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:54,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:53:54,996 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:53:54,996 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:54,996 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:54,996 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,996 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:54,998 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,998 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:54,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,998 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:54,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:54,999 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:54,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:54,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:54,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:54,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:54,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:55,000 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:55,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:55,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:55,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,004 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:55,004 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,004 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:55,004 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:55,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:55,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:55,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:55,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:55,007 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,007 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:55,008 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,008 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,008 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,008 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:55,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:55,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:55,008 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:53:55,009 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:55,009 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:55,109 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:55,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:55,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:55,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:55,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,121 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:55,122 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:55,122 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:53:55,123 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:53:55,123 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,123 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,125 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:55,125 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:55,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:55,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:55,126 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:55,126 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:55,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:55,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,127 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:53:55,128 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:55,128 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:53:55,128 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:55,128 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:53:55,128 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:55,129 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:55,129 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:55,129 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. started... 2023-03-24 21:53:55,129 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 46db6a74ea50377062351d8426481e30 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:55,560 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f3de13d81b2044aeae41f0096cc03f84 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205f3de13d81b2044aeae41f0096cc03f84 2023-03-24 21:53:55,996 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/f10c2a5f0f1d4d04a6da8fc80464c792 2023-03-24 21:53:56,005 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/f10c2a5f0f1d4d04a6da8fc80464c792 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/f10c2a5f0f1d4d04a6da8fc80464c792 2023-03-24 21:53:56,011 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/f10c2a5f0f1d4d04a6da8fc80464c792, entries=1, sequenceid=17, filesize=4.9 K 2023-03-24 21:53:56,012 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 46db6a74ea50377062351d8426481e30 in 883ms, sequenceid=17, compaction requested=true 2023-03-24 21:53:56,013 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:56,013 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:56,013 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:56,013 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:56,013 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,013 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:53:56,013 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:53:56,016 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,016 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:56,016 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,017 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:56,019 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:53:56,019 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b144467[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:56,019 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:53:56,019 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,020 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,020 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:53:56,020 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:53:56,020 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:56,021 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,021 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:56,022 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,022 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:53:56,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:56,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:56,022 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:56,022 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:53:56,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:56,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:56,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:56,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:56,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:53:56,027 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:53:56,027 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:56,027 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:56,027 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,027 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:56,029 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,029 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:56,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:56,029 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:56,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:56,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:56,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:56,033 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:56,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,034 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:56,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,039 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:56,039 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:56,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:56,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:56,039 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,039 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,039 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,040 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,040 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,040 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,040 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,041 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:56,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:56,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:53:56,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:56,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:56,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:56,044 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:56,045 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:56,145 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:56,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:56,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:56,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:56,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,183 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:56,183 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:56,187 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:53:56,187 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:53:56,191 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,191 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,192 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:56,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:56,193 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:56,193 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,193 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:56,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,194 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,195 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:53:56,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,195 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:53:56,195 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:56,196 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:53:56,196 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:56,196 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:56,196 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:56,196 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. started... 2023-03-24 21:53:56,196 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 46db6a74ea50377062351d8426481e30 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:56,254 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228c143f74883a746f8b60eaa30518ff36a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228c143f74883a746f8b60eaa30518ff36a 2023-03-24 21:53:56,279 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/d4b2cffbbc0442c8b016f5ef5cd6fcb3 2023-03-24 21:53:56,289 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/d4b2cffbbc0442c8b016f5ef5cd6fcb3 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d4b2cffbbc0442c8b016f5ef5cd6fcb3 2023-03-24 21:53:56,297 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d4b2cffbbc0442c8b016f5ef5cd6fcb3, entries=1, sequenceid=21, filesize=4.9 K 2023-03-24 21:53:56,299 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 46db6a74ea50377062351d8426481e30 in 103ms, sequenceid=21, compaction requested=true 2023-03-24 21:53:56,299 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:56,299 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:56,299 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:56,299 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:56,299 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,299 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:53:56,299 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:53:56,302 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,302 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:56,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:56,304 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,304 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:56,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:56,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:53:56,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21979761[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:56,308 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:53:56,308 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,310 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,311 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:53:56,311 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:53:56,311 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:56,311 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,311 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:56,312 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:56,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:56,313 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:53:56,313 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:56,313 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:53:56,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:56,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:56,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:56,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:56,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:53:56,316 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:53:56,316 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:56,316 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:56,316 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,316 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:56,322 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,322 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:56,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:56,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,322 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:56,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:56,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:56,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:56,323 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:56,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,325 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:56,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,329 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,329 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,329 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,329 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:56,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:56,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:56,329 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,329 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,329 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,329 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:53:56,330 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:56,331 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:56,331 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:56,331 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:56,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:56,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:56,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:56,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:56,431 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:56,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:56,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:56,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:56,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,443 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:56,443 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:56,443 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:53:56,444 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:53:56,444 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,444 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,446 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:56,446 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:56,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:56,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,447 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,447 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:56,448 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:56,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,448 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,449 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:53:56,449 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,449 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:53:56,449 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:56,450 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:53:56,450 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:56,451 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:56,451 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:56,451 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. started... 2023-03-24 21:53:56,451 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 46db6a74ea50377062351d8426481e30 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:56,480 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512319e8da3308794463c81539f244f84e6e0 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512319e8da3308794463c81539f244f84e6e0 2023-03-24 21:53:56,889 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/78f051a5558848e2be09a6fdd1d8a26f 2023-03-24 21:53:56,903 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/78f051a5558848e2be09a6fdd1d8a26f as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/78f051a5558848e2be09a6fdd1d8a26f 2023-03-24 21:53:56,911 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/78f051a5558848e2be09a6fdd1d8a26f, entries=1, sequenceid=25, filesize=4.9 K 2023-03-24 21:53:56,912 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 46db6a74ea50377062351d8426481e30 in 461ms, sequenceid=25, compaction requested=true 2023-03-24 21:53:56,912 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:56,912 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:56,912 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:56,913 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:56,913 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,913 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:53:56,913 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:53:56,915 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,915 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:56,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:56,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:56,915 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,916 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:56,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:56,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:56,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:53:56,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7fc501fd[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:56,917 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:53:56,917 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,920 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,920 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:53:56,920 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:53:56,920 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:56,920 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,921 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:56,922 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,922 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:53:56,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:56,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:56,922 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:56,922 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:53:56,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:56,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:56,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:56,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:56,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:53:56,927 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:53:56,927 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:56,927 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:56,927 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,928 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:56,930 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,931 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:56,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,930 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:56,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:56,931 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:56,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:56,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:56,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:56,933 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:56,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,936 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:56,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:56,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:56,937 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:56,937 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:56,937 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:56,937 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:56,937 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,937 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:56,937 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,937 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:56,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:56,937 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,937 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,937 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:56,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:56,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:56,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:53:57,037 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:57,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:57,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:57,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:57,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,052 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:57,052 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:57,053 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:53:57,053 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:53:57,053 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,053 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,056 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:57,056 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:57,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:57,057 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,057 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:57,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:57,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:53:57,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,058 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:53:57,058 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:57,059 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:53:57,059 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:57,059 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:57,059 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:57,059 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. started... 2023-03-24 21:53:57,059 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 46db6a74ea50377062351d8426481e30 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:57,085 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010185ac0f212a0b49569638b69af8562eb2 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010185ac0f212a0b49569638b69af8562eb2 2023-03-24 21:53:57,094 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/d883435dfac549c88ddae9c3b9e1d8d1 2023-03-24 21:53:57,101 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/d883435dfac549c88ddae9c3b9e1d8d1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d883435dfac549c88ddae9c3b9e1d8d1 2023-03-24 21:53:57,108 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d883435dfac549c88ddae9c3b9e1d8d1, entries=1, sequenceid=29, filesize=4.9 K 2023-03-24 21:53:57,110 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 46db6a74ea50377062351d8426481e30 in 51ms, sequenceid=29, compaction requested=true 2023-03-24 21:53:57,110 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:57,110 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:57,110 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:57,110 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:57,110 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,110 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:53:57,110 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:53:57,112 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,112 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:57,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:57,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:57,113 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,113 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:57,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:57,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:57,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:53:57,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59df63af[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:57,114 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:53:57,115 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,116 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,116 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:53:57,116 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:53:57,116 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:57,116 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,117 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:57,118 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:57,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:57,118 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:53:57,118 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:57,118 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:53:57,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:57,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:57,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:57,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:57,122 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:53:57,122 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:53:57,122 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:57,122 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:57,122 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,122 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:57,125 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,125 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,125 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:57,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:57,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:57,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:57,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:57,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:57,126 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:57,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,128 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:57,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,131 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,131 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:57,131 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:57,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:57,131 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:57,131 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,131 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,131 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,131 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,132 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,131 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:57,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:57,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:57,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:57,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:57,133 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:57,133 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:57,154 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-24 21:53:57,233 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:57,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:57,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:57,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:57,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,250 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:57,250 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:57,252 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:53:57,252 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:53:57,255 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,255 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,257 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:57,257 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:57,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:57,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,258 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,258 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:57,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:57,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:53:57,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,259 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:53:57,259 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:57,261 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:53:57,271 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:57,271 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:57,271 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:57,271 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. started... 2023-03-24 21:53:57,271 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 46db6a74ea50377062351d8426481e30 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:57,294 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103caeee065702a45df92e7a5c6af9f1dab to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103caeee065702a45df92e7a5c6af9f1dab 2023-03-24 21:53:57,300 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/d02e183fa40a4cf69c627dfe0ba135c7 2023-03-24 21:53:57,309 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/d02e183fa40a4cf69c627dfe0ba135c7 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d02e183fa40a4cf69c627dfe0ba135c7 2023-03-24 21:53:57,316 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d02e183fa40a4cf69c627dfe0ba135c7, entries=1, sequenceid=33, filesize=4.9 K 2023-03-24 21:53:57,318 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 46db6a74ea50377062351d8426481e30 in 47ms, sequenceid=33, compaction requested=true 2023-03-24 21:53:57,318 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:57,318 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:57,318 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:57,318 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:57,318 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,318 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:53:57,318 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:53:57,320 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:57,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:57,321 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:57,322 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:57,322 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:57,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:57,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:53:57,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5061338f[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:57,325 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:53:57,325 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,326 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,327 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:53:57,327 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:53:57,327 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:57,327 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,327 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:57,332 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,332 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:53:57,332 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:57,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,332 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:53:57,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:57,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:57,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:57,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:57,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:57,336 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:53:57,336 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:53:57,336 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:57,336 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:57,336 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,336 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:57,337 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,337 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,337 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:57,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,338 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:57,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:57,341 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:57,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,343 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,343 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,343 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,343 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:57,343 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:57,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:57,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:57,343 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,344 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,344 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,344 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:57,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:57,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:57,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:57,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:53:57,344 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:57,343 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:57,344 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:57,445 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:57,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:57,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:57,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:57,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,458 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:57,458 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:57,459 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:53:57,459 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:53:57,460 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,460 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,462 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,462 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:57,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:57,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:57,462 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,462 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:57,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,463 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:57,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,463 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,463 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:53:57,463 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,463 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:53:57,463 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:57,464 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:53:57,464 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:57,464 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:57,464 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:57,464 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. started... 2023-03-24 21:53:57,465 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 46db6a74ea50377062351d8426481e30 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:57,491 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907d5e861fda4f646f6af2cc386adbc06e8 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20150907d5e861fda4f646f6af2cc386adbc06e8 2023-03-24 21:53:57,499 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/cc61bf60c86443fc9bf56f0898143a61 2023-03-24 21:53:57,506 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/cc61bf60c86443fc9bf56f0898143a61 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/cc61bf60c86443fc9bf56f0898143a61 2023-03-24 21:53:57,511 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/cc61bf60c86443fc9bf56f0898143a61, entries=1, sequenceid=37, filesize=4.9 K 2023-03-24 21:53:57,512 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 46db6a74ea50377062351d8426481e30 in 47ms, sequenceid=37, compaction requested=true 2023-03-24 21:53:57,513 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:57,513 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:57,513 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:57,513 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:57,513 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,513 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:53:57,513 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:53:57,514 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,514 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:57,515 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:57,515 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:57,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:53:57,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d508a7d[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:57,517 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:53:57,517 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,518 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,518 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:53:57,518 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:53:57,518 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:57,518 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,518 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:57,521 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,521 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:53:57,521 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:57,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:57,521 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:53:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:57,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:57,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:57,526 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:53:57,526 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:53:57,526 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:57,526 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:57,526 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,526 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:57,528 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,528 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,528 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:57,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:57,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:57,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:57,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:57,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:57,529 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:57,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,531 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:57,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,533 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,533 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:57,533 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:57,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:57,534 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:57,534 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,534 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,534 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,534 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,534 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,534 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:57,534 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:53:57,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:57,535 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:57,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:57,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:57,535 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:57,535 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:57,636 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:57,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:57,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:53:57,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:53:57,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,645 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:57,645 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:57,646 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:53:57,646 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:53:57,646 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,646 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:57,648 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:57,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:57,648 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:57,648 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:53:57,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,648 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:53:57,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,649 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,649 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:53:57,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:57,649 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:53:57,649 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:53:57,650 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:53:57,651 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:53:57,651 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:53:57,651 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:57,651 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. started... 2023-03-24 21:53:57,651 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 46db6a74ea50377062351d8426481e30 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:53:57,669 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511204a1206e01e27407fb912d2a1fd0f888c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511204a1206e01e27407fb912d2a1fd0f888c 2023-03-24 21:53:58,075 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/91e20564aab747aa9e8cdf403bb0fe57 2023-03-24 21:53:58,083 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/91e20564aab747aa9e8cdf403bb0fe57 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/91e20564aab747aa9e8cdf403bb0fe57 2023-03-24 21:53:58,089 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/91e20564aab747aa9e8cdf403bb0fe57, entries=1, sequenceid=41, filesize=4.9 K 2023-03-24 21:53:58,090 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 46db6a74ea50377062351d8426481e30 in 439ms, sequenceid=41, compaction requested=true 2023-03-24 21:53:58,091 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:58,091 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:58,091 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:53:58,091 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:53:58,091 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,091 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:53:58,091 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:53:58,093 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,093 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:58,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:58,094 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,094 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:53:58,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:58,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:58,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:58,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:53:58,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a4da0c3[Count = 0] remaining members to acquire global barrier 2023-03-24 21:53:58,096 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:53:58,096 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,097 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,097 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:53:58,097 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:53:58,097 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:53:58,097 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,097 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:53:58,099 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:53:58,099 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,099 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:58,099 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:53:58,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:58,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:58,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:58,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:58,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:53:58,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:53:58,104 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:53:58,104 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:53:58,104 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:53:58,104 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,104 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:53:58,106 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,106 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:53:58,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:53:58,106 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:58,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:53:58,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:58,106 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:58,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:53:58,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,108 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:53:58,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,111 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,111 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,111 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,111 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:53:58,111 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,111 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:53:58,111 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:53:58,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-24 21:53:58,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:53:58,111 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:53:58,113 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:53:58,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:53:58,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:53:58,113 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,114 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:53:58,114 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:53:58,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:53:58,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:53:58,214 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:53:58,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:53:58,218 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-03-24 21:53:58,221 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-24 21:53:58,222 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-24 21:53:58,225 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695438225 2023-03-24 21:53:58,226 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:53:58,227 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-24 21:53:58,228 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-24 21:53:58,228 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:53:58,229 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x139197bb to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:53:58,257 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@558ceda, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:53:58,257 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:53:58,728 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010372e47a78fe084da48d4f91c5766471f2 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010372e47a78fe084da48d4f91c5766471f2 2023-03-24 21:53:58,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:53:58,746 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:53:58,748 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:49434, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:53:58,758 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:53:58,759 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:46804, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:53:58,772 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/97377ea72f9a46849c74b14112a2923f first=Optional[r5] last=Optional[r6] 2023-03-24 21:53:58,773 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/97377ea72f9a46849c74b14112a2923f}] 2023-03-24 21:53:58,778 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/97377ea72f9a46849c74b14112a2923f for inclusion in 46db6a74ea50377062351d8426481e30/family1 2023-03-24 21:53:58,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-24 21:53:58,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:53:58,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:58,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/97377ea72f9a46849c74b14112a2923f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__99vdft8cut1k7i4coaqo2qjlo1c5k9l75ve65epaj9k2e96bnn625hnjdlojv2us/family1/97377ea72f9a46849c74b14112a2923f 2023-03-24 21:53:58,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__99vdft8cut1k7i4coaqo2qjlo1c5k9l75ve65epaj9k2e96bnn625hnjdlojv2us/family1/97377ea72f9a46849c74b14112a2923f as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/a05c11616d304616996d289a990986e3_SeqId_44_ 2023-03-24 21:53:58,787 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__99vdft8cut1k7i4coaqo2qjlo1c5k9l75ve65epaj9k2e96bnn625hnjdlojv2us/family1/97377ea72f9a46849c74b14112a2923f into 46db6a74ea50377062351d8426481e30/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/a05c11616d304616996d289a990986e3_SeqId_44_ - updating store file list. 2023-03-24 21:53:58,793 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/a05c11616d304616996d289a990986e3_SeqId_44_ into 46db6a74ea50377062351d8426481e30/family1 2023-03-24 21:53:58,794 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__99vdft8cut1k7i4coaqo2qjlo1c5k9l75ve65epaj9k2e96bnn625hnjdlojv2us/family1/97377ea72f9a46849c74b14112a2923f into 46db6a74ea50377062351d8426481e30/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/a05c11616d304616996d289a990986e3_SeqId_44_) 2023-03-24 21:53:58,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__99vdft8cut1k7i4coaqo2qjlo1c5k9l75ve65epaj9k2e96bnn625hnjdlojv2us/family1/97377ea72f9a46849c74b14112a2923f 2023-03-24 21:53:58,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-24 21:53:58,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-24 21:53:58,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(1912): 46db6a74ea50377062351d8426481e30/family1 is initiating minor compaction 2023-03-24 21:53:58,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6d0713d9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:53:58,800 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 46db6a74ea50377062351d8426481e30/family1 in testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:58,800 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/2b2de07382b54595897ea641e98c5e01, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/094ad5929e5f4c048ea457912fd5029d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/f10c2a5f0f1d4d04a6da8fc80464c792, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d4b2cffbbc0442c8b016f5ef5cd6fcb3, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/78f051a5558848e2be09a6fdd1d8a26f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d883435dfac549c88ddae9c3b9e1d8d1, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d02e183fa40a4cf69c627dfe0ba135c7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/cc61bf60c86443fc9bf56f0898143a61, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/91e20564aab747aa9e8cdf403bb0fe57, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/a05c11616d304616996d289a990986e3_SeqId_44_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp, totalSize=49.7 K 2023-03-24 21:53:58,800 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 2b2de07382b54595897ea641e98c5e01, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-24 21:53:58,800 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 094ad5929e5f4c048ea457912fd5029d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-24 21:53:58,800 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting f10c2a5f0f1d4d04a6da8fc80464c792, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-24 21:53:58,800 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting d4b2cffbbc0442c8b016f5ef5cd6fcb3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-24 21:53:58,801 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 78f051a5558848e2be09a6fdd1d8a26f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-24 21:53:58,801 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting d883435dfac549c88ddae9c3b9e1d8d1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-24 21:53:58,801 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting d02e183fa40a4cf69c627dfe0ba135c7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-24 21:53:58,801 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting cc61bf60c86443fc9bf56f0898143a61, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-24 21:53:58,801 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 91e20564aab747aa9e8cdf403bb0fe57, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-24 21:53:58,801 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting a05c11616d304616996d289a990986e3_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-24 21:53:58,803 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:53:58,805 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:53:58,808 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010185ac0f212a0b49569638b69af8562eb2 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010185ac0f212a0b49569638b69af8562eb2 2023-03-24 21:53:58,809 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103caeee065702a45df92e7a5c6af9f1dab to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103caeee065702a45df92e7a5c6af9f1dab 2023-03-24 21:53:58,809 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-24 21:53:58,809 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-24 21:53:58,810 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:53:58,810 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:53:58,874 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): 46db6a74ea50377062351d8426481e30#family1#compaction#22 average throughput is 0.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:53:59,323 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/4b40a9b68a4c43d8855ac38d0e61bd45 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/4b40a9b68a4c43d8855ac38d0e61bd45 2023-03-24 21:53:59,339 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 46db6a74ea50377062351d8426481e30/family1 of 46db6a74ea50377062351d8426481e30 into 4b40a9b68a4c43d8855ac38d0e61bd45(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:53:59,340 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:59,340 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1679694838799; duration=0sec 2023-03-24 21:53:59,340 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:53:59,373 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/2b2de07382b54595897ea641e98c5e01, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/094ad5929e5f4c048ea457912fd5029d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/f10c2a5f0f1d4d04a6da8fc80464c792, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d4b2cffbbc0442c8b016f5ef5cd6fcb3, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/78f051a5558848e2be09a6fdd1d8a26f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d883435dfac549c88ddae9c3b9e1d8d1, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d02e183fa40a4cf69c627dfe0ba135c7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/cc61bf60c86443fc9bf56f0898143a61, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/91e20564aab747aa9e8cdf403bb0fe57, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/a05c11616d304616996d289a990986e3_SeqId_44_] to archive 2023-03-24 21:53:59,375 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:53:59,377 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/2b2de07382b54595897ea641e98c5e01 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/2b2de07382b54595897ea641e98c5e01 2023-03-24 21:53:59,379 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/094ad5929e5f4c048ea457912fd5029d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/094ad5929e5f4c048ea457912fd5029d 2023-03-24 21:53:59,382 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/f10c2a5f0f1d4d04a6da8fc80464c792 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/f10c2a5f0f1d4d04a6da8fc80464c792 2023-03-24 21:53:59,383 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d4b2cffbbc0442c8b016f5ef5cd6fcb3 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d4b2cffbbc0442c8b016f5ef5cd6fcb3 2023-03-24 21:53:59,385 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/78f051a5558848e2be09a6fdd1d8a26f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/78f051a5558848e2be09a6fdd1d8a26f 2023-03-24 21:53:59,386 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d883435dfac549c88ddae9c3b9e1d8d1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d883435dfac549c88ddae9c3b9e1d8d1 2023-03-24 21:53:59,388 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d02e183fa40a4cf69c627dfe0ba135c7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/d02e183fa40a4cf69c627dfe0ba135c7 2023-03-24 21:53:59,390 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/cc61bf60c86443fc9bf56f0898143a61 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/cc61bf60c86443fc9bf56f0898143a61 2023-03-24 21:53:59,392 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/91e20564aab747aa9e8cdf403bb0fe57 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/91e20564aab747aa9e8cdf403bb0fe57 2023-03-24 21:53:59,394 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/a05c11616d304616996d289a990986e3_SeqId_44_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/a05c11616d304616996d289a990986e3_SeqId_44_ 2023-03-24 21:53:59,724 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231366129cc394949b7b7a66e9c0b90a6b5 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231366129cc394949b7b7a66e9c0b90a6b5 2023-03-24 21:53:59,725 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:53:59,753 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/0694549268f647d8bb1feba5564ca30e first=Optional[r1] last=Optional[r4] 2023-03-24 21:53:59,753 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/0694549268f647d8bb1feba5564ca30e}] 2023-03-24 21:53:59,756 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/0694549268f647d8bb1feba5564ca30e for inclusion in 46db6a74ea50377062351d8426481e30/family1 2023-03-24 21:53:59,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-24 21:53:59,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:53:59,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:53:59,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/0694549268f647d8bb1feba5564ca30e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__3ui4ueuuk2d74570g69o3aknjjm8cjtasgob74atqvljbo07ssdppcfgbf8rf3j5/family1/0694549268f647d8bb1feba5564ca30e 2023-03-24 21:53:59,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__3ui4ueuuk2d74570g69o3aknjjm8cjtasgob74atqvljbo07ssdppcfgbf8rf3j5/family1/0694549268f647d8bb1feba5564ca30e as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/ec2063e47c9942d3abed2878f8ad7190_SeqId_47_ 2023-03-24 21:53:59,764 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__3ui4ueuuk2d74570g69o3aknjjm8cjtasgob74atqvljbo07ssdppcfgbf8rf3j5/family1/0694549268f647d8bb1feba5564ca30e into 46db6a74ea50377062351d8426481e30/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/ec2063e47c9942d3abed2878f8ad7190_SeqId_47_ - updating store file list. 2023-03-24 21:53:59,768 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/ec2063e47c9942d3abed2878f8ad7190_SeqId_47_ into 46db6a74ea50377062351d8426481e30/family1 2023-03-24 21:53:59,769 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__3ui4ueuuk2d74570g69o3aknjjm8cjtasgob74atqvljbo07ssdppcfgbf8rf3j5/family1/0694549268f647d8bb1feba5564ca30e into 46db6a74ea50377062351d8426481e30/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/ec2063e47c9942d3abed2878f8ad7190_SeqId_47_) 2023-03-24 21:53:59,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__3ui4ueuuk2d74570g69o3aknjjm8cjtasgob74atqvljbo07ssdppcfgbf8rf3j5/family1/0694549268f647d8bb1feba5564ca30e 2023-03-24 21:53:59,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-24 21:53:59,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-24 21:53:59,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(1912): 46db6a74ea50377062351d8426481e30/family1 is initiating minor compaction (all files) 2023-03-24 21:53:59,771 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 46db6a74ea50377062351d8426481e30/family1 in testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:53:59,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4db45c54; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:53:59,771 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/20e26bd69914450ba8d5d9e247730c01, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/4b40a9b68a4c43d8855ac38d0e61bd45, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/ec2063e47c9942d3abed2878f8ad7190_SeqId_47_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp, totalSize=178.1 K 2023-03-24 21:53:59,772 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 20e26bd69914450ba8d5d9e247730c01, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841999 2023-03-24 21:53:59,772 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 4b40a9b68a4c43d8855ac38d0e61bd45, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241999 2023-03-24 21:53:59,773 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting ec2063e47c9942d3abed2878f8ad7190_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965241999 2023-03-24 21:53:59,775 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:53:59,777 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:53:59,779 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512010fc16511e4dd4598b50900b93d9e6097 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512010fc16511e4dd4598b50900b93d9e6097 2023-03-24 21:53:59,780 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205f3de13d81b2044aeae41f0096cc03f84 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205f3de13d81b2044aeae41f0096cc03f84 2023-03-24 21:53:59,782 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228c143f74883a746f8b60eaa30518ff36a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228c143f74883a746f8b60eaa30518ff36a 2023-03-24 21:53:59,784 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512319e8da3308794463c81539f244f84e6e0 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512319e8da3308794463c81539f244f84e6e0 2023-03-24 21:53:59,784 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-24 21:53:59,784 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-24 21:53:59,785 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:53:59,785 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:53:59,797 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): 46db6a74ea50377062351d8426481e30#family1#compaction#23 average throughput is 14.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:54:00,309 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/.tmp/family1/0b6a71cc16994889bdbf2ae89ecb46e6 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/0b6a71cc16994889bdbf2ae89ecb46e6 2023-03-24 21:54:00,318 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 46db6a74ea50377062351d8426481e30/family1 of 46db6a74ea50377062351d8426481e30 into 0b6a71cc16994889bdbf2ae89ecb46e6(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:54:00,318 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:54:00,318 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1679694839771; duration=0sec 2023-03-24 21:54:00,319 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:00,351 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d13ca7265f8c4a12b23cf5d4b36f7051 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130d13ca7265f8c4a12b23cf5d4b36f7051 2023-03-24 21:54:00,352 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:54:00,364 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/20e26bd69914450ba8d5d9e247730c01, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/4b40a9b68a4c43d8855ac38d0e61bd45, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/ec2063e47c9942d3abed2878f8ad7190_SeqId_47_] to archive 2023-03-24 21:54:00,365 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:00,368 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/20e26bd69914450ba8d5d9e247730c01 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/20e26bd69914450ba8d5d9e247730c01 2023-03-24 21:54:00,370 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/4b40a9b68a4c43d8855ac38d0e61bd45 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/4b40a9b68a4c43d8855ac38d0e61bd45 2023-03-24 21:54:00,372 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/ec2063e47c9942d3abed2878f8ad7190_SeqId_47_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/ec2063e47c9942d3abed2878f8ad7190_SeqId_47_ 2023-03-24 21:54:00,405 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/bb3d57c171334afca5fae91fb4ce86a6 first=Optional[r00] last=Optional[r8] 2023-03-24 21:54:00,406 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/bb3d57c171334afca5fae91fb4ce86a6}] 2023-03-24 21:54:00,409 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/bb3d57c171334afca5fae91fb4ce86a6 for inclusion in 46db6a74ea50377062351d8426481e30/family1 2023-03-24 21:54:00,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-24 21:54:00,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:54:00,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:54:00,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/bb3d57c171334afca5fae91fb4ce86a6 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1m5ou667qouh2bgi1dvmunajetqnv4bb49tpos35ijcqt67u26imi29dbk2rscd6/family1/bb3d57c171334afca5fae91fb4ce86a6 2023-03-24 21:54:00,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1m5ou667qouh2bgi1dvmunajetqnv4bb49tpos35ijcqt67u26imi29dbk2rscd6/family1/bb3d57c171334afca5fae91fb4ce86a6 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/6a621412725246c8a9f9a950d32be5c1_SeqId_50_ 2023-03-24 21:54:00,422 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1m5ou667qouh2bgi1dvmunajetqnv4bb49tpos35ijcqt67u26imi29dbk2rscd6/family1/bb3d57c171334afca5fae91fb4ce86a6 into 46db6a74ea50377062351d8426481e30/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/6a621412725246c8a9f9a950d32be5c1_SeqId_50_ - updating store file list. 2023-03-24 21:54:00,433 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/6a621412725246c8a9f9a950d32be5c1_SeqId_50_ into 46db6a74ea50377062351d8426481e30/family1 2023-03-24 21:54:00,433 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1m5ou667qouh2bgi1dvmunajetqnv4bb49tpos35ijcqt67u26imi29dbk2rscd6/family1/bb3d57c171334afca5fae91fb4ce86a6 into 46db6a74ea50377062351d8426481e30/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/family1/6a621412725246c8a9f9a950d32be5c1_SeqId_50_) 2023-03-24 21:54:00,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1m5ou667qouh2bgi1dvmunajetqnv4bb49tpos35ijcqt67u26imi29dbk2rscd6/family1/bb3d57c171334afca5fae91fb4ce86a6 2023-03-24 21:54:00,437 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:54:00,438 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:00,440 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511204a1206e01e27407fb912d2a1fd0f888c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511204a1206e01e27407fb912d2a1fd0f888c 2023-03-24 21:54:00,441 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128295679f4a15a4915aa57297a4b4a061a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128295679f4a15a4915aa57297a4b4a061a 2023-03-24 21:54:00,443 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511306c710908e0a245249dc07a66b7a6981e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511306c710908e0a245249dc07a66b7a6981e 2023-03-24 21:54:00,443 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-24 21:54:00,443 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-24 21:54:00,444 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:54:00,444 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:54:00,444 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:54:00,444 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:54:00,444 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:00,446 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-24 21:54:00,448 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.2270 sec 2023-03-24 21:54:00,582 INFO [Listener at localhost/39907] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=260 (was 254) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55192 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2aaccc5e-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:51980 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:52014 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:52058 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55162 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2aaccc5e-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/cluster_90ef77c7-dcec-9426-4d35-a9af4e7f4c97/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:52068 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:52082 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55346 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2aaccc5e-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_869545651_15 at /127.0.0.1:52098 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:52100 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55428 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:51978 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5ea55c48-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2aaccc5e-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54680@0x139197bb-SendThread(127.0.0.1:54680) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54680@0x139197bb sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/168963814.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_869545651_15 at /127.0.0.1:52032 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2aaccc5e-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:52020 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54680@0x139197bb-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2aaccc5e-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2aaccc5e-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55374 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:52012 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_779047479_15 at /127.0.0.1:55314 [Waiting for operation #39] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=625 (was 601) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=403 (was 377) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 170), AvailableMemoryMB=8440 (was 8538) 2023-03-24 21:54:00,609 INFO [Listener at localhost/39907] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=259, OpenFileDescriptor=623, MaxFileDescriptor=60000, SystemLoadAverage=403, ProcessCount=168, AvailableMemoryMB=8439 2023-03-24 21:54:00,613 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-24 21:54:00,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:00,616 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:54:00,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-03-24 21:54:00,618 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:54:00,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-24 21:54:00,621 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887 2023-03-24 21:54:00,627 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887 empty. 2023-03-24 21:54:00,628 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887 2023-03-24 21:54:00,628 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-03-24 21:54:00,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-24 21:54:00,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-24 21:54:01,078 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-24 21:54:01,079 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 11f0b79484a112e4405e43efbfd3e887, NAME => 'testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:54:01,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-24 21:54:01,490 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:01,490 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 11f0b79484a112e4405e43efbfd3e887, disabling compactions & flushes 2023-03-24 21:54:01,490 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:01,490 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:01,490 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. after waiting 0 ms 2023-03-24 21:54:01,490 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:01,490 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:01,490 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:01,493 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:54:01,494 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679694841494"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694841494"}]},"ts":"1679694841494"} 2023-03-24 21:54:01,496 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-24 21:54:01,497 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:54:01,497 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694841497"}]},"ts":"1679694841497"} 2023-03-24 21:54:01,499 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-24 21:54:01,505 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=11f0b79484a112e4405e43efbfd3e887, ASSIGN}] 2023-03-24 21:54:01,508 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=11f0b79484a112e4405e43efbfd3e887, ASSIGN 2023-03-24 21:54:01,509 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=11f0b79484a112e4405e43efbfd3e887, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:54:01,660 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=11f0b79484a112e4405e43efbfd3e887, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:01,660 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679694841660"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694841660"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694841660"}]},"ts":"1679694841660"} 2023-03-24 21:54:01,662 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 11f0b79484a112e4405e43efbfd3e887, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:01,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-24 21:54:01,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:01,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 11f0b79484a112e4405e43efbfd3e887, NAME => 'testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:54:01,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 11f0b79484a112e4405e43efbfd3e887 2023-03-24 21:54:01,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:01,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 11f0b79484a112e4405e43efbfd3e887 2023-03-24 21:54:01,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 11f0b79484a112e4405e43efbfd3e887 2023-03-24 21:54:01,821 INFO [StoreOpener-11f0b79484a112e4405e43efbfd3e887-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 11f0b79484a112e4405e43efbfd3e887 2023-03-24 21:54:01,824 DEBUG [StoreOpener-11f0b79484a112e4405e43efbfd3e887-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1 2023-03-24 21:54:01,824 DEBUG [StoreOpener-11f0b79484a112e4405e43efbfd3e887-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1 2023-03-24 21:54:01,824 INFO [StoreOpener-11f0b79484a112e4405e43efbfd3e887-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 11f0b79484a112e4405e43efbfd3e887 columnFamilyName family1 2023-03-24 21:54:01,827 INFO [StoreOpener-11f0b79484a112e4405e43efbfd3e887-1] regionserver.HStore(310): Store=11f0b79484a112e4405e43efbfd3e887/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:01,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887 2023-03-24 21:54:01,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887 2023-03-24 21:54:01,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 11f0b79484a112e4405e43efbfd3e887 2023-03-24 21:54:01,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:54:01,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 11f0b79484a112e4405e43efbfd3e887; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68738213, jitterRate=0.024279192090034485}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:54:01,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:01,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887., pid=25, masterSystemTime=1679694841815 2023-03-24 21:54:01,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:01,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:01,841 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=11f0b79484a112e4405e43efbfd3e887, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:01,841 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679694841841"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694841841"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694841841"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694841841"}]},"ts":"1679694841841"} 2023-03-24 21:54:01,846 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-03-24 21:54:01,846 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 11f0b79484a112e4405e43efbfd3e887, server=jenkins-hbase9.apache.org,41807,1679694819103 in 181 msec 2023-03-24 21:54:01,848 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-03-24 21:54:01,848 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=11f0b79484a112e4405e43efbfd3e887, ASSIGN in 341 msec 2023-03-24 21:54:01,849 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:54:01,849 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694841849"}]},"ts":"1679694841849"} 2023-03-24 21:54:01,850 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-24 21:54:01,853 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:54:01,854 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.2400 sec 2023-03-24 21:54:02,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-24 21:54:02,725 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-03-24 21:54:02,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:02,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,781 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:02,781 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:02,781 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:02,781 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:02,781 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,781 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,783 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,783 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:02,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:02,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:02,783 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,783 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,784 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,784 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,785 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:02,785 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:02,785 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:02,786 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:02,786 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:02,786 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:02,786 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. started... 2023-03-24 21:54:02,787 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 11f0b79484a112e4405e43efbfd3e887 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-24 21:54:02,830 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130973d7a1d787f47d293ba23606f885de8 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130973d7a1d787f47d293ba23606f885de8 2023-03-24 21:54:02,838 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/c9cbed81f3d643e79bc7a99a59c4852d 2023-03-24 21:54:02,849 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/c9cbed81f3d643e79bc7a99a59c4852d as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/c9cbed81f3d643e79bc7a99a59c4852d 2023-03-24 21:54:02,862 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/c9cbed81f3d643e79bc7a99a59c4852d, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-24 21:54:02,863 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 11f0b79484a112e4405e43efbfd3e887 in 76ms, sequenceid=5, compaction requested=false 2023-03-24 21:54:02,864 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:02,864 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:02,864 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:02,864 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:02,864 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:02,865 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,865 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:02,865 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:02,867 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,867 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:02,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:02,867 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:02,867 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:02,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:02,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:02,869 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:02,869 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38ad6a3e[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:02,869 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:02,869 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,870 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,871 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:02,871 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:02,871 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:02,871 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,871 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:02,874 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,874 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:02,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,874 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:02,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:02,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:02,874 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:02,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:02,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:02,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:02,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:02,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:02,878 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:02,878 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:02,878 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:02,878 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,878 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:02,880 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,880 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:02,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,880 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:02,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:02,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:02,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:02,881 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:02,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:02,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,884 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:02,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,886 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,887 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:02,887 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,887 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:02,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:02,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:02,887 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,887 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:02,887 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,887 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,887 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:02,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:02,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:02,887 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:02,887 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:02,888 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:02,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:02,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:02,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:02,988 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:02,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:02,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:03,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:03,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,001 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:03,001 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:03,001 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:03,001 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:03,001 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,001 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,002 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:03,002 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:03,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:03,003 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,003 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:03,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:03,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,004 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,008 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:03,008 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,008 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:03,008 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:03,008 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:03,009 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:03,009 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:03,009 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:03,009 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. started... 2023-03-24 21:54:03,009 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 11f0b79484a112e4405e43efbfd3e887 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-24 21:54:03,044 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511281a1b9730206843e68493ba8d4262b988 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511281a1b9730206843e68493ba8d4262b988 2023-03-24 21:54:03,457 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/becd31d5538545758584a01e8e521f15 2023-03-24 21:54:03,469 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/becd31d5538545758584a01e8e521f15 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/becd31d5538545758584a01e8e521f15 2023-03-24 21:54:03,476 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/becd31d5538545758584a01e8e521f15, entries=1, sequenceid=9, filesize=5.0 K 2023-03-24 21:54:03,478 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 11f0b79484a112e4405e43efbfd3e887 in 469ms, sequenceid=9, compaction requested=false 2023-03-24 21:54:03,479 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:03,479 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:03,479 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:03,479 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:03,479 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,479 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:03,479 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:03,481 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,481 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:03,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:03,482 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,482 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:03,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:03,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:03,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:03,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:03,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6109cbef[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:03,484 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:03,484 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,485 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,486 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:03,486 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,486 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:03,486 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:03,486 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:03,488 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:03,488 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,488 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:03,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:03,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:03,488 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:03,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:03,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:03,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:03,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:03,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:03,492 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:03,492 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:03,492 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:03,492 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,492 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:03,499 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,499 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:03,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,499 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:03,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:03,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:03,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:03,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:03,504 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,505 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:03,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:03,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:03,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:03,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:03,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,508 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:03,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:03,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:03,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:03,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,509 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:03,510 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:03,610 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:03,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:03,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:03,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:03,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,618 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:03,618 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:03,618 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:03,618 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:03,618 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,618 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,624 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,624 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:03,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:03,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:03,624 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,624 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:03,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,625 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,625 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,625 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:03,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,626 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:03,626 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:03,626 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:03,626 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:03,626 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:03,626 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:03,626 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. started... 2023-03-24 21:54:03,626 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 11f0b79484a112e4405e43efbfd3e887 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:03,646 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201705b52e67cfa4de39e1e4f506d6bffb8 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201705b52e67cfa4de39e1e4f506d6bffb8 2023-03-24 21:54:03,665 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/e25aa049fa6845f6a436edfa8b401bec 2023-03-24 21:54:03,671 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/e25aa049fa6845f6a436edfa8b401bec as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/e25aa049fa6845f6a436edfa8b401bec 2023-03-24 21:54:03,678 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/e25aa049fa6845f6a436edfa8b401bec, entries=1, sequenceid=13, filesize=4.9 K 2023-03-24 21:54:03,680 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 11f0b79484a112e4405e43efbfd3e887 in 54ms, sequenceid=13, compaction requested=true 2023-03-24 21:54:03,680 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:03,680 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:03,680 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:03,680 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:03,680 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,680 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:03,680 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:03,682 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,682 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,682 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:03,683 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:03,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:03,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:03,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:03,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:03,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:03,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28c3b6d1[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:03,685 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:03,685 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,687 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,687 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:03,687 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:03,687 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:03,688 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,688 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:03,689 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,689 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:03,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:03,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:03,689 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:03,689 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:03,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:03,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:03,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:03,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:03,691 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:03,691 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:03,691 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:03,691 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:03,691 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,691 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:03,693 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,693 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,693 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:03,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:03,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:03,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:03,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:03,694 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:03,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:03,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,695 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:03,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,701 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,701 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,701 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,701 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,701 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:03,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:03,701 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:03,701 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:03,701 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:03,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:03,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:03,701 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:03,701 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:03,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:03,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:03,702 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:03,701 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,702 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,801 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:03,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:03,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:03,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:03,809 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,809 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:03,809 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:03,809 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:03,809 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:03,809 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,809 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,811 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:03,811 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:03,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:03,811 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:03,811 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:03,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,812 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:03,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,812 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,812 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:03,812 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:03,813 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:03,813 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:03,813 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:03,813 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:03,813 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:03,813 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:03,813 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. started... 2023-03-24 21:54:03,813 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 11f0b79484a112e4405e43efbfd3e887 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:03,833 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120528db425359484657afc242c0a957a89c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120528db425359484657afc242c0a957a89c 2023-03-24 21:54:04,240 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/7b44e919eef841d38aa9724c80b049bf 2023-03-24 21:54:04,245 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/7b44e919eef841d38aa9724c80b049bf as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/7b44e919eef841d38aa9724c80b049bf 2023-03-24 21:54:04,250 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/7b44e919eef841d38aa9724c80b049bf, entries=1, sequenceid=17, filesize=4.9 K 2023-03-24 21:54:04,251 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 11f0b79484a112e4405e43efbfd3e887 in 438ms, sequenceid=17, compaction requested=true 2023-03-24 21:54:04,251 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:04,251 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:04,251 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:04,252 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:04,252 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,252 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:04,252 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:04,253 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,253 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:04,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:04,254 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,254 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:04,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:04,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:04,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:04,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:04,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f7f40b2[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:04,255 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:04,255 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,256 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,256 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:04,256 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:04,257 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,257 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:04,257 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:04,258 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,258 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:04,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:04,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:04,258 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:04,258 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:04,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:04,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:04,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:04,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:04,260 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:04,260 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:04,260 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:04,260 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:04,260 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,261 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:04,262 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,262 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:04,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:04,263 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:04,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,263 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:04,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:04,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:04,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:04,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,264 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:04,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,267 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,267 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:04,267 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,267 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:04,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:04,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:04,267 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:04,267 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:04,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:04,267 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:04,267 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,267 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:04,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:04,267 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,268 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,268 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:04,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:04,368 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:04,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:04,372 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:04,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:04,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,374 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:04,374 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:04,374 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:04,374 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:04,374 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,374 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,379 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:04,380 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:04,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:04,380 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:04,380 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:04,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,380 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:04,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,381 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,381 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:04,381 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:04,381 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:04,381 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:04,381 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:04,381 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:04,381 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:04,381 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:04,381 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. started... 2023-03-24 21:54:04,381 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 11f0b79484a112e4405e43efbfd3e887 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:04,795 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228daac3820a46a451e9f137958b7832cbd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228daac3820a46a451e9f137958b7832cbd 2023-03-24 21:54:05,201 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/d6b8218006f24fd6bce1d8abee7044f2 2023-03-24 21:54:05,207 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/d6b8218006f24fd6bce1d8abee7044f2 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/d6b8218006f24fd6bce1d8abee7044f2 2023-03-24 21:54:05,212 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/d6b8218006f24fd6bce1d8abee7044f2, entries=1, sequenceid=21, filesize=4.9 K 2023-03-24 21:54:05,213 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 11f0b79484a112e4405e43efbfd3e887 in 832ms, sequenceid=21, compaction requested=true 2023-03-24 21:54:05,213 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:05,213 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:05,213 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:05,213 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:05,213 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,213 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:05,213 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:05,215 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,215 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:05,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:05,215 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,215 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:05,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:05,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:05,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:05,217 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:05,217 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c503df9[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:05,217 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:05,217 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,219 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,219 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:05,219 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:05,219 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:05,219 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,219 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:05,222 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,222 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:05,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:05,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:05,222 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:05,223 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:05,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:05,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:05,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:05,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:05,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:05,225 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:05,225 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:05,225 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:05,225 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,225 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:05,226 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,226 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:05,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:05,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,226 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:05,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:05,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:05,227 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:05,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,228 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:05,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,230 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,230 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,230 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,230 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:05,230 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:05,230 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:05,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:05,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:05,230 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:05,230 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:05,231 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,231 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,231 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,231 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:05,231 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:05,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:05,331 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:05,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:05,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:05,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:05,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,339 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:05,339 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:05,339 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:05,339 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:05,339 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,339 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,341 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,341 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:05,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:05,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:05,342 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:05,342 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:05,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:05,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:05,343 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:05,343 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:05,343 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:05,343 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:05,343 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:05,343 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:05,343 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:05,343 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. started... 2023-03-24 21:54:05,343 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 11f0b79484a112e4405e43efbfd3e887 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:05,759 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316c3533dbe901405bb57cadb1e9a7f6a5 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512316c3533dbe901405bb57cadb1e9a7f6a5 2023-03-24 21:54:06,165 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/93ed5a6ae69c4d22a52f6cd4b9c8787c 2023-03-24 21:54:06,170 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/93ed5a6ae69c4d22a52f6cd4b9c8787c as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/93ed5a6ae69c4d22a52f6cd4b9c8787c 2023-03-24 21:54:06,175 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/93ed5a6ae69c4d22a52f6cd4b9c8787c, entries=1, sequenceid=25, filesize=4.9 K 2023-03-24 21:54:06,176 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 11f0b79484a112e4405e43efbfd3e887 in 833ms, sequenceid=25, compaction requested=true 2023-03-24 21:54:06,176 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:06,176 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:06,176 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:06,176 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:06,176 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,177 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:06,177 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:06,178 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,178 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:06,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:06,178 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,178 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:06,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:06,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:06,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23a9d92f[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:06,180 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:06,180 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,181 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,181 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:06,181 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,181 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:06,181 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:06,181 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:06,183 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,183 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:06,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:06,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:06,183 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:06,183 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:06,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:06,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:06,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:06,185 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:06,185 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:06,185 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:06,185 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,185 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:06,190 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,190 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:06,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:06,190 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:06,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,191 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:06,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:06,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:06,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:06,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,192 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:06,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,194 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,194 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,194 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:06,194 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:06,194 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:06,194 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:06,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:06,194 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:06,194 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,194 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:06,195 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:06,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:06,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:06,195 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:06,194 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,195 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,195 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:06,295 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:06,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:06,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:06,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:06,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,302 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:06,302 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:06,302 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:06,302 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:06,302 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,302 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,304 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,304 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:06,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:06,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:06,304 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:06,304 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:06,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,305 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:06,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,305 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,305 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:06,305 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:06,305 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:06,305 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:06,306 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:06,306 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:06,306 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:06,306 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:06,306 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. started... 2023-03-24 21:54:06,306 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 11f0b79484a112e4405e43efbfd3e887 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:06,722 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601017a1a607a5a94494ba7276e76c33fe137 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601017a1a607a5a94494ba7276e76c33fe137 2023-03-24 21:54:07,128 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/18555c16be584b10905df0eaed84c4fa 2023-03-24 21:54:07,134 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/18555c16be584b10905df0eaed84c4fa as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/18555c16be584b10905df0eaed84c4fa 2023-03-24 21:54:07,138 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/18555c16be584b10905df0eaed84c4fa, entries=1, sequenceid=29, filesize=4.9 K 2023-03-24 21:54:07,139 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 11f0b79484a112e4405e43efbfd3e887 in 833ms, sequenceid=29, compaction requested=true 2023-03-24 21:54:07,139 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:07,139 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:07,139 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:07,140 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:07,140 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,140 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:07,140 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:07,141 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,141 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:07,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:07,142 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:07,142 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:07,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:07,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:07,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:07,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16e986b5[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:07,143 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:07,143 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,144 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,144 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:07,144 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:07,144 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:07,144 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,144 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:07,145 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:07,145 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:07,145 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,145 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:07,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:07,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:07,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:07,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:07,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:07,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:07,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:07,148 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:07,148 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:07,148 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:07,148 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,148 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:07,150 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,150 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:07,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:07,150 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:07,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:07,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:07,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:07,151 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:07,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,152 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:07,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,154 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,154 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:07,154 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,154 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:07,154 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:07,154 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:07,154 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,154 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:07,154 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:07,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:07,155 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:07,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:07,155 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:07,155 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:07,255 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:07,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:07,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:07,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:07,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,262 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:07,262 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:07,262 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:07,262 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:07,262 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,262 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,264 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,264 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:07,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:07,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:07,264 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:07,264 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:07,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:07,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,265 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,265 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:07,265 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:07,265 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:07,265 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:07,265 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:07,265 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:07,265 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:07,266 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:07,266 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. started... 2023-03-24 21:54:07,266 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 11f0b79484a112e4405e43efbfd3e887 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:07,321 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-24 21:54:07,679 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d252ddd15d484e03a44875b86bc7900c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103d252ddd15d484e03a44875b86bc7900c 2023-03-24 21:54:08,086 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/a695dcaf38df4a3eb73f8ab5001864ea 2023-03-24 21:54:08,091 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/a695dcaf38df4a3eb73f8ab5001864ea as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/a695dcaf38df4a3eb73f8ab5001864ea 2023-03-24 21:54:08,096 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/a695dcaf38df4a3eb73f8ab5001864ea, entries=1, sequenceid=33, filesize=4.9 K 2023-03-24 21:54:08,097 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 11f0b79484a112e4405e43efbfd3e887 in 831ms, sequenceid=33, compaction requested=true 2023-03-24 21:54:08,097 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:08,097 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:08,097 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:08,097 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:08,097 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,097 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:08,097 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:08,101 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,101 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:08,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:08,101 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:08,101 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:08,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:08,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:08,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:08,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33b63e0a[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:08,102 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:08,103 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,104 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,104 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:08,104 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:08,104 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:08,104 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,104 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:08,105 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,105 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:08,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:08,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:08,105 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:08,106 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:08,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:08,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:08,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:08,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:08,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:08,108 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:08,108 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:08,108 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:08,108 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,108 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:08,109 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,109 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:08,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:08,109 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:08,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,110 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:08,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,111 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,114 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,114 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,114 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:08,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:08,114 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:08,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:08,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:08,114 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:08,114 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:08,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:08,114 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,115 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,115 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,115 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:08,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:08,115 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:08,115 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:08,215 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:08,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:08,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:08,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:08,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,222 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:08,222 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:08,222 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:08,222 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:08,223 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,223 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,224 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,224 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:08,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:08,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:08,224 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:08,224 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:08,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:08,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,225 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:08,225 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:08,226 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:08,226 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:08,226 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:08,226 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:08,226 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:08,226 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:08,226 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. started... 2023-03-24 21:54:08,226 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 11f0b79484a112e4405e43efbfd3e887 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:08,639 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907bfbedc4df10043db9f801ae5a609f039 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907bfbedc4df10043db9f801ae5a609f039 2023-03-24 21:54:09,045 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/1698105deb2a46dbb1bae6eb57b11167 2023-03-24 21:54:09,051 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/1698105deb2a46dbb1bae6eb57b11167 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/1698105deb2a46dbb1bae6eb57b11167 2023-03-24 21:54:09,055 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/1698105deb2a46dbb1bae6eb57b11167, entries=1, sequenceid=37, filesize=4.9 K 2023-03-24 21:54:09,056 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 11f0b79484a112e4405e43efbfd3e887 in 830ms, sequenceid=37, compaction requested=true 2023-03-24 21:54:09,056 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:09,056 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:09,056 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:09,056 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:09,057 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,057 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:09,057 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:09,058 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,058 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:09,058 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,058 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:09,060 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:09,060 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e846270[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:09,060 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:09,060 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,065 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,065 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,065 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:09,065 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:09,065 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:09,065 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:09,067 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,067 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:09,067 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:09,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:09,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:09,067 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:09,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:09,070 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:09,070 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:09,070 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:09,070 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:09,070 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,070 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:09,072 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,072 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,072 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:09,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:09,072 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,073 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,076 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,076 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:09,076 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:09,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:09,076 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:09,076 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:09,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:09,076 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:09,076 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:09,076 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,076 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:09,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:09,076 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:09,077 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,077 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:09,077 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,177 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:09,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:09,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:09,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:09,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,184 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:09,184 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:09,184 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:09,185 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:09,185 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,185 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,187 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:09,187 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:09,188 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:09,188 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,188 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:09,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,189 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,189 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:09,189 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:09,189 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:09,189 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:09,189 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:09,189 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:09,189 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:09,189 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:09,189 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. started... 2023-03-24 21:54:09,190 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 11f0b79484a112e4405e43efbfd3e887 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:09,603 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a3497023d981440282b0855e0558411d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120a3497023d981440282b0855e0558411d 2023-03-24 21:54:10,010 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/48576930cc324d439cdd9d58258e7df6 2023-03-24 21:54:10,015 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/48576930cc324d439cdd9d58258e7df6 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/48576930cc324d439cdd9d58258e7df6 2023-03-24 21:54:10,020 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/48576930cc324d439cdd9d58258e7df6, entries=1, sequenceid=41, filesize=4.9 K 2023-03-24 21:54:10,021 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 11f0b79484a112e4405e43efbfd3e887 in 831ms, sequenceid=41, compaction requested=true 2023-03-24 21:54:10,021 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:10,021 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:10,021 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:10,021 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:10,021 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,021 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:10,021 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:10,023 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,023 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:10,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:10,023 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,023 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:10,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:10,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:10,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:10,024 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:10,024 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5309a42e[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:10,024 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:10,024 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,025 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,026 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:10,026 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,026 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:10,026 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:10,026 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:10,028 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:10,028 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,028 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:10,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:10,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:10,028 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:10,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:10,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:10,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:10,030 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:10,030 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:10,030 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:10,031 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,031 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:10,032 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,032 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:10,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,032 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:10,033 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,034 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,036 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,036 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:10,036 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:10,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-24 21:54:10,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:10,036 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:10,036 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:10,036 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:10,036 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:10,036 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:10,036 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:10,036 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,036 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:10,036 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,136 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:10,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:10,139 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-03-24 21:54:10,141 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-24 21:54:10,142 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-24 21:54:10,143 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:10,143 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695450143 2023-03-24 21:54:10,144 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-24 21:54:10,145 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-24 21:54:10,145 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-24 21:54:10,146 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x705970c9 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:54:10,154 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@33b45e06, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:54:10,154 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:10,997 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ed8e5b6eef4c4308843465fd975c5b37 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103ed8e5b6eef4c4308843465fd975c5b37 2023-03-24 21:54:10,998 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-24 21:54:11,010 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:54:11,012 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:53432, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:54:11,019 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:54:11,021 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:41556, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:54:11,030 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dc31f6b6a8db4474af56071ec6a56950 first=Optional[r3] last=Optional[r6] 2023-03-24 21:54:11,031 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dc31f6b6a8db4474af56071ec6a56950}] 2023-03-24 21:54:11,036 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dc31f6b6a8db4474af56071ec6a56950 for inclusion in 11f0b79484a112e4405e43efbfd3e887/family1 2023-03-24 21:54:11,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-24 21:54:11,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:54:11,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:11,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dc31f6b6a8db4474af56071ec6a56950 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7letq0046ui7jua24p1mr9lr7de8dp3qnpj329hvb2nmmjqdc5t2li8vj15hjse8/family1/dc31f6b6a8db4474af56071ec6a56950 2023-03-24 21:54:11,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7letq0046ui7jua24p1mr9lr7de8dp3qnpj329hvb2nmmjqdc5t2li8vj15hjse8/family1/dc31f6b6a8db4474af56071ec6a56950 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/97334f6a685b4d56b281822b43f58f4f_SeqId_44_ 2023-03-24 21:54:11,044 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7letq0046ui7jua24p1mr9lr7de8dp3qnpj329hvb2nmmjqdc5t2li8vj15hjse8/family1/dc31f6b6a8db4474af56071ec6a56950 into 11f0b79484a112e4405e43efbfd3e887/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/97334f6a685b4d56b281822b43f58f4f_SeqId_44_ - updating store file list. 2023-03-24 21:54:11,049 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/97334f6a685b4d56b281822b43f58f4f_SeqId_44_ into 11f0b79484a112e4405e43efbfd3e887/family1 2023-03-24 21:54:11,049 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7letq0046ui7jua24p1mr9lr7de8dp3qnpj329hvb2nmmjqdc5t2li8vj15hjse8/family1/dc31f6b6a8db4474af56071ec6a56950 into 11f0b79484a112e4405e43efbfd3e887/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/97334f6a685b4d56b281822b43f58f4f_SeqId_44_) 2023-03-24 21:54:11,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7letq0046ui7jua24p1mr9lr7de8dp3qnpj329hvb2nmmjqdc5t2li8vj15hjse8/family1/dc31f6b6a8db4474af56071ec6a56950 2023-03-24 21:54:11,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-24 21:54:11,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-24 21:54:11,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(1912): 11f0b79484a112e4405e43efbfd3e887/family1 is initiating minor compaction 2023-03-24 21:54:11,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@43dd06; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:11,053 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 11f0b79484a112e4405e43efbfd3e887/family1 in testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:11,053 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/becd31d5538545758584a01e8e521f15, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/e25aa049fa6845f6a436edfa8b401bec, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/7b44e919eef841d38aa9724c80b049bf, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/d6b8218006f24fd6bce1d8abee7044f2, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/93ed5a6ae69c4d22a52f6cd4b9c8787c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/18555c16be584b10905df0eaed84c4fa, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/a695dcaf38df4a3eb73f8ab5001864ea, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/1698105deb2a46dbb1bae6eb57b11167, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/48576930cc324d439cdd9d58258e7df6, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/97334f6a685b4d56b281822b43f58f4f_SeqId_44_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp, totalSize=50.0 K 2023-03-24 21:54:11,053 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting becd31d5538545758584a01e8e521f15, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-24 21:54:11,053 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting e25aa049fa6845f6a436edfa8b401bec, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-24 21:54:11,053 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 7b44e919eef841d38aa9724c80b049bf, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-24 21:54:11,053 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting d6b8218006f24fd6bce1d8abee7044f2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-24 21:54:11,054 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 93ed5a6ae69c4d22a52f6cd4b9c8787c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-24 21:54:11,054 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 18555c16be584b10905df0eaed84c4fa, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-24 21:54:11,054 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting a695dcaf38df4a3eb73f8ab5001864ea, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-24 21:54:11,054 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 1698105deb2a46dbb1bae6eb57b11167, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-24 21:54:11,054 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 48576930cc324d439cdd9d58258e7df6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-24 21:54:11,054 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 97334f6a685b4d56b281822b43f58f4f_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-24 21:54:11,055 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-24 21:54:11,056 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:11,058 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228daac3820a46a451e9f137958b7832cbd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228daac3820a46a451e9f137958b7832cbd 2023-03-24 21:54:11,059 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512316c3533dbe901405bb57cadb1e9a7f6a5 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512316c3533dbe901405bb57cadb1e9a7f6a5 2023-03-24 21:54:11,061 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601017a1a607a5a94494ba7276e76c33fe137 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601017a1a607a5a94494ba7276e76c33fe137 2023-03-24 21:54:11,062 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103d252ddd15d484e03a44875b86bc7900c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103d252ddd15d484e03a44875b86bc7900c 2023-03-24 21:54:11,062 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-24 21:54:11,062 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-24 21:54:11,062 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:11,062 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:11,062 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:11,063 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:11,063 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:11,063 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:11,063 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:11,063 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:11,083 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): 11f0b79484a112e4405e43efbfd3e887#family1#compaction#34 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:54:11,895 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120520338443518b41bb80716e697dd40c1e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120520338443518b41bb80716e697dd40c1e 2023-03-24 21:54:11,896 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-24 21:54:11,898 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/a314f860ef5e409281a75df2baac19cd as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/a314f860ef5e409281a75df2baac19cd 2023-03-24 21:54:11,904 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 11f0b79484a112e4405e43efbfd3e887/family1 of 11f0b79484a112e4405e43efbfd3e887 into a314f860ef5e409281a75df2baac19cd(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:54:11,904 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:11,904 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1679694851053; duration=0sec 2023-03-24 21:54:11,904 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:11,927 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c2d0a3ab5fc2447998ff39b5236c4d3a first=Optional[r1] last=Optional[r2] 2023-03-24 21:54:11,928 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c2d0a3ab5fc2447998ff39b5236c4d3a}] 2023-03-24 21:54:11,931 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c2d0a3ab5fc2447998ff39b5236c4d3a for inclusion in 11f0b79484a112e4405e43efbfd3e887/family1 2023-03-24 21:54:11,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-24 21:54:11,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:54:11,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:11,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c2d0a3ab5fc2447998ff39b5236c4d3a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__gtre41j0i67iap5kde5jdrisok9ugmmcp6mrf67338eu5te5fdoiu83v7sv95qfe/family1/c2d0a3ab5fc2447998ff39b5236c4d3a 2023-03-24 21:54:11,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__gtre41j0i67iap5kde5jdrisok9ugmmcp6mrf67338eu5te5fdoiu83v7sv95qfe/family1/c2d0a3ab5fc2447998ff39b5236c4d3a as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/09f392ed75544439a4553328cf1504ba_SeqId_47_ 2023-03-24 21:54:11,938 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__gtre41j0i67iap5kde5jdrisok9ugmmcp6mrf67338eu5te5fdoiu83v7sv95qfe/family1/c2d0a3ab5fc2447998ff39b5236c4d3a into 11f0b79484a112e4405e43efbfd3e887/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/09f392ed75544439a4553328cf1504ba_SeqId_47_ - updating store file list. 2023-03-24 21:54:11,942 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/09f392ed75544439a4553328cf1504ba_SeqId_47_ into 11f0b79484a112e4405e43efbfd3e887/family1 2023-03-24 21:54:11,942 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__gtre41j0i67iap5kde5jdrisok9ugmmcp6mrf67338eu5te5fdoiu83v7sv95qfe/family1/c2d0a3ab5fc2447998ff39b5236c4d3a into 11f0b79484a112e4405e43efbfd3e887/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/09f392ed75544439a4553328cf1504ba_SeqId_47_) 2023-03-24 21:54:11,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__gtre41j0i67iap5kde5jdrisok9ugmmcp6mrf67338eu5te5fdoiu83v7sv95qfe/family1/c2d0a3ab5fc2447998ff39b5236c4d3a 2023-03-24 21:54:11,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-24 21:54:11,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-24 21:54:11,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(1912): 11f0b79484a112e4405e43efbfd3e887/family1 is initiating minor compaction (all files) 2023-03-24 21:54:11,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@71e7ac97; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:11,944 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 11f0b79484a112e4405e43efbfd3e887/family1 in testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:54:11,944 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/c9cbed81f3d643e79bc7a99a59c4852d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/a314f860ef5e409281a75df2baac19cd, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/09f392ed75544439a4553328cf1504ba_SeqId_47_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp, totalSize=176.8 K 2023-03-24 21:54:11,945 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting c9cbed81f3d643e79bc7a99a59c4852d, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841999 2023-03-24 21:54:11,945 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting a314f860ef5e409281a75df2baac19cd, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241999 2023-03-24 21:54:11,946 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 09f392ed75544439a4553328cf1504ba_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965241999 2023-03-24 21:54:11,946 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-24 21:54:11,947 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:11,949 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201705b52e67cfa4de39e1e4f506d6bffb8 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201705b52e67cfa4de39e1e4f506d6bffb8 2023-03-24 21:54:11,950 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120528db425359484657afc242c0a957a89c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120528db425359484657afc242c0a957a89c 2023-03-24 21:54:11,950 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-03-24 21:54:11,950 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-24 21:54:11,951 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:11,951 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-24 21:54:11,951 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:11,952 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-24 21:54:11,954 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8120 sec 2023-03-24 21:54:11,965 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): 11f0b79484a112e4405e43efbfd3e887#family1#compaction#35 average throughput is 17.18 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:54:11,980 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/becd31d5538545758584a01e8e521f15, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/e25aa049fa6845f6a436edfa8b401bec, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/7b44e919eef841d38aa9724c80b049bf, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/d6b8218006f24fd6bce1d8abee7044f2, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/93ed5a6ae69c4d22a52f6cd4b9c8787c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/18555c16be584b10905df0eaed84c4fa, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/a695dcaf38df4a3eb73f8ab5001864ea, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/1698105deb2a46dbb1bae6eb57b11167, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/48576930cc324d439cdd9d58258e7df6, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/97334f6a685b4d56b281822b43f58f4f_SeqId_44_] to archive 2023-03-24 21:54:11,981 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:11,983 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/becd31d5538545758584a01e8e521f15 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/becd31d5538545758584a01e8e521f15 2023-03-24 21:54:11,984 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/e25aa049fa6845f6a436edfa8b401bec to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/e25aa049fa6845f6a436edfa8b401bec 2023-03-24 21:54:11,988 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/7b44e919eef841d38aa9724c80b049bf to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/7b44e919eef841d38aa9724c80b049bf 2023-03-24 21:54:11,989 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/d6b8218006f24fd6bce1d8abee7044f2 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/d6b8218006f24fd6bce1d8abee7044f2 2023-03-24 21:54:11,991 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/93ed5a6ae69c4d22a52f6cd4b9c8787c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/93ed5a6ae69c4d22a52f6cd4b9c8787c 2023-03-24 21:54:11,992 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/18555c16be584b10905df0eaed84c4fa to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/18555c16be584b10905df0eaed84c4fa 2023-03-24 21:54:11,993 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/a695dcaf38df4a3eb73f8ab5001864ea to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/a695dcaf38df4a3eb73f8ab5001864ea 2023-03-24 21:54:11,995 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/1698105deb2a46dbb1bae6eb57b11167 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/1698105deb2a46dbb1bae6eb57b11167 2023-03-24 21:54:11,996 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/48576930cc324d439cdd9d58258e7df6 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/48576930cc324d439cdd9d58258e7df6 2023-03-24 21:54:11,997 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 29 more 2023-03-24 21:54:11,997 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/97334f6a685b4d56b281822b43f58f4f_SeqId_44_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/97334f6a685b4d56b281822b43f58f4f_SeqId_44_ 2023-03-24 21:54:12,064 INFO [Listener at localhost/39907] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=266 (was 259) - Thread LEAK? -, OpenFileDescriptor=623 (was 623), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=341 (was 403), ProcessCount=169 (was 168) - ProcessCount LEAK? -, AvailableMemoryMB=8373 (was 8439) 2023-03-24 21:54:12,077 INFO [Listener at localhost/39907] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=266, OpenFileDescriptor=623, MaxFileDescriptor=60000, SystemLoadAverage=341, ProcessCount=169, AvailableMemoryMB=8372 2023-03-24 21:54:12,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-24 21:54:12,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:12,082 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:54:12,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-03-24 21:54:12,083 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:54:12,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-24 21:54:12,086 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a7163a0a63017bfa630fabe660e05be6 2023-03-24 21:54:12,086 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c 2023-03-24 21:54:12,086 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4467112a96b2d78b9da907dc14725abd 2023-03-24 21:54:12,086 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a7163a0a63017bfa630fabe660e05be6 empty. 2023-03-24 21:54:12,086 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c empty. 2023-03-24 21:54:12,086 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4467112a96b2d78b9da907dc14725abd empty. 2023-03-24 21:54:12,087 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a7163a0a63017bfa630fabe660e05be6 2023-03-24 21:54:12,087 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4467112a96b2d78b9da907dc14725abd 2023-03-24 21:54:12,087 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c 2023-03-24 21:54:12,087 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-03-24 21:54:12,095 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-03-24 21:54:12,098 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => 3080bc8c83bb721d4a0167e2a8e9a57c, NAME => 'testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:54:12,098 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => a7163a0a63017bfa630fabe660e05be6, NAME => 'testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:54:12,098 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => 4467112a96b2d78b9da907dc14725abd, NAME => 'testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:54:12,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-24 21:54:12,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-24 21:54:12,514 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:12,514 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:12,514 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing a7163a0a63017bfa630fabe660e05be6, disabling compactions & flushes 2023-03-24 21:54:12,514 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:12,514 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing 4467112a96b2d78b9da907dc14725abd, disabling compactions & flushes 2023-03-24 21:54:12,514 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing 3080bc8c83bb721d4a0167e2a8e9a57c, disabling compactions & flushes 2023-03-24 21:54:12,514 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:54:12,514 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:54:12,514 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:54:12,514 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. after waiting 0 ms 2023-03-24 21:54:12,514 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:54:12,514 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:54:12,514 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:54:12,514 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:54:12,514 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:54:12,515 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for 4467112a96b2d78b9da907dc14725abd: 2023-03-24 21:54:12,514 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. after waiting 0 ms 2023-03-24 21:54:12,515 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:54:12,515 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. after waiting 0 ms 2023-03-24 21:54:12,515 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:54:12,515 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:54:12,515 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for 3080bc8c83bb721d4a0167e2a8e9a57c: 2023-03-24 21:54:12,515 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:54:12,515 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for a7163a0a63017bfa630fabe660e05be6: 2023-03-24 21:54:12,517 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:54:12,518 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1679694852518"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694852518"}]},"ts":"1679694852518"} 2023-03-24 21:54:12,518 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679694852518"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694852518"}]},"ts":"1679694852518"} 2023-03-24 21:54:12,518 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679694852518"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694852518"}]},"ts":"1679694852518"} 2023-03-24 21:54:12,521 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-24 21:54:12,522 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:54:12,522 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694852522"}]},"ts":"1679694852522"} 2023-03-24 21:54:12,523 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-03-24 21:54:12,526 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a7163a0a63017bfa630fabe660e05be6, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4467112a96b2d78b9da907dc14725abd, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3080bc8c83bb721d4a0167e2a8e9a57c, ASSIGN}] 2023-03-24 21:54:12,528 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a7163a0a63017bfa630fabe660e05be6, ASSIGN 2023-03-24 21:54:12,528 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4467112a96b2d78b9da907dc14725abd, ASSIGN 2023-03-24 21:54:12,528 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3080bc8c83bb721d4a0167e2a8e9a57c, ASSIGN 2023-03-24 21:54:12,529 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a7163a0a63017bfa630fabe660e05be6, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:54:12,529 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4467112a96b2d78b9da907dc14725abd, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:54:12,529 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3080bc8c83bb721d4a0167e2a8e9a57c, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:54:12,681 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4467112a96b2d78b9da907dc14725abd, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:12,681 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=3080bc8c83bb721d4a0167e2a8e9a57c, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:12,681 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=a7163a0a63017bfa630fabe660e05be6, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:12,681 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1679694852681"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694852681"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694852681"}]},"ts":"1679694852681"} 2023-03-24 21:54:12,681 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679694852681"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694852681"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694852681"}]},"ts":"1679694852681"} 2023-03-24 21:54:12,681 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679694852681"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694852681"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694852681"}]},"ts":"1679694852681"} 2023-03-24 21:54:12,683 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=29, state=RUNNABLE; OpenRegionProcedure 4467112a96b2d78b9da907dc14725abd, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:12,684 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=28, state=RUNNABLE; OpenRegionProcedure a7163a0a63017bfa630fabe660e05be6, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:12,684 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=30, state=RUNNABLE; OpenRegionProcedure 3080bc8c83bb721d4a0167e2a8e9a57c, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:12,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-24 21:54:12,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:54:12,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a7163a0a63017bfa630fabe660e05be6, NAME => 'testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6.', STARTKEY => '', ENDKEY => '1'} 2023-03-24 21:54:12,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles a7163a0a63017bfa630fabe660e05be6 2023-03-24 21:54:12,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:12,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for a7163a0a63017bfa630fabe660e05be6 2023-03-24 21:54:12,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for a7163a0a63017bfa630fabe660e05be6 2023-03-24 21:54:12,867 INFO [StoreOpener-a7163a0a63017bfa630fabe660e05be6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a7163a0a63017bfa630fabe660e05be6 2023-03-24 21:54:12,872 DEBUG [StoreOpener-a7163a0a63017bfa630fabe660e05be6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/a7163a0a63017bfa630fabe660e05be6/family1 2023-03-24 21:54:12,872 DEBUG [StoreOpener-a7163a0a63017bfa630fabe660e05be6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/a7163a0a63017bfa630fabe660e05be6/family1 2023-03-24 21:54:12,873 INFO [StoreOpener-a7163a0a63017bfa630fabe660e05be6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a7163a0a63017bfa630fabe660e05be6 columnFamilyName family1 2023-03-24 21:54:12,876 INFO [StoreOpener-a7163a0a63017bfa630fabe660e05be6-1] regionserver.HStore(310): Store=a7163a0a63017bfa630fabe660e05be6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:12,876 INFO [StoreOpener-a7163a0a63017bfa630fabe660e05be6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a7163a0a63017bfa630fabe660e05be6 2023-03-24 21:54:12,881 DEBUG [StoreOpener-a7163a0a63017bfa630fabe660e05be6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/a7163a0a63017bfa630fabe660e05be6/family2 2023-03-24 21:54:12,881 DEBUG [StoreOpener-a7163a0a63017bfa630fabe660e05be6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/a7163a0a63017bfa630fabe660e05be6/family2 2023-03-24 21:54:12,882 INFO [StoreOpener-a7163a0a63017bfa630fabe660e05be6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a7163a0a63017bfa630fabe660e05be6 columnFamilyName family2 2023-03-24 21:54:12,888 INFO [StoreOpener-a7163a0a63017bfa630fabe660e05be6-1] regionserver.HStore(310): Store=a7163a0a63017bfa630fabe660e05be6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:12,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/a7163a0a63017bfa630fabe660e05be6 2023-03-24 21:54:12,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/a7163a0a63017bfa630fabe660e05be6 2023-03-24 21:54:12,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:54:12,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for a7163a0a63017bfa630fabe660e05be6 2023-03-24 21:54:12,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/a7163a0a63017bfa630fabe660e05be6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:54:12,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened a7163a0a63017bfa630fabe660e05be6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59087147, jitterRate=-0.11953289806842804}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:54:12,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for a7163a0a63017bfa630fabe660e05be6: 2023-03-24 21:54:12,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6., pid=32, masterSystemTime=1679694852837 2023-03-24 21:54:12,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:54:12,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:54:12,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:54:12,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3080bc8c83bb721d4a0167e2a8e9a57c, NAME => 'testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c.', STARTKEY => '2', ENDKEY => ''} 2023-03-24 21:54:12,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 3080bc8c83bb721d4a0167e2a8e9a57c 2023-03-24 21:54:12,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:12,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 3080bc8c83bb721d4a0167e2a8e9a57c 2023-03-24 21:54:12,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 3080bc8c83bb721d4a0167e2a8e9a57c 2023-03-24 21:54:12,906 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=a7163a0a63017bfa630fabe660e05be6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:12,906 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679694852906"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694852906"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694852906"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694852906"}]},"ts":"1679694852906"} 2023-03-24 21:54:12,911 INFO [StoreOpener-3080bc8c83bb721d4a0167e2a8e9a57c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3080bc8c83bb721d4a0167e2a8e9a57c 2023-03-24 21:54:12,911 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-03-24 21:54:12,911 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; OpenRegionProcedure a7163a0a63017bfa630fabe660e05be6, server=jenkins-hbase9.apache.org,41807,1679694819103 in 224 msec 2023-03-24 21:54:12,913 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a7163a0a63017bfa630fabe660e05be6, ASSIGN in 385 msec 2023-03-24 21:54:12,913 DEBUG [StoreOpener-3080bc8c83bb721d4a0167e2a8e9a57c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1 2023-03-24 21:54:12,913 DEBUG [StoreOpener-3080bc8c83bb721d4a0167e2a8e9a57c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1 2023-03-24 21:54:12,914 INFO [StoreOpener-3080bc8c83bb721d4a0167e2a8e9a57c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3080bc8c83bb721d4a0167e2a8e9a57c columnFamilyName family1 2023-03-24 21:54:12,917 INFO [StoreOpener-3080bc8c83bb721d4a0167e2a8e9a57c-1] regionserver.HStore(310): Store=3080bc8c83bb721d4a0167e2a8e9a57c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:12,917 INFO [StoreOpener-3080bc8c83bb721d4a0167e2a8e9a57c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 3080bc8c83bb721d4a0167e2a8e9a57c 2023-03-24 21:54:12,920 DEBUG [StoreOpener-3080bc8c83bb721d4a0167e2a8e9a57c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family2 2023-03-24 21:54:12,920 DEBUG [StoreOpener-3080bc8c83bb721d4a0167e2a8e9a57c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family2 2023-03-24 21:54:12,921 INFO [StoreOpener-3080bc8c83bb721d4a0167e2a8e9a57c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3080bc8c83bb721d4a0167e2a8e9a57c columnFamilyName family2 2023-03-24 21:54:12,922 INFO [StoreOpener-3080bc8c83bb721d4a0167e2a8e9a57c-1] regionserver.HStore(310): Store=3080bc8c83bb721d4a0167e2a8e9a57c/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:12,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c 2023-03-24 21:54:12,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c 2023-03-24 21:54:12,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:54:12,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 3080bc8c83bb721d4a0167e2a8e9a57c 2023-03-24 21:54:12,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:54:12,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 3080bc8c83bb721d4a0167e2a8e9a57c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72435415, jitterRate=0.07937179505825043}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:54:12,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 3080bc8c83bb721d4a0167e2a8e9a57c: 2023-03-24 21:54:12,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c., pid=33, masterSystemTime=1679694852837 2023-03-24 21:54:12,955 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:54:12,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:54:12,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:54:12,955 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4467112a96b2d78b9da907dc14725abd, NAME => 'testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd.', STARTKEY => '1', ENDKEY => '2'} 2023-03-24 21:54:12,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 4467112a96b2d78b9da907dc14725abd 2023-03-24 21:54:12,956 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=3080bc8c83bb721d4a0167e2a8e9a57c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:12,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:12,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 4467112a96b2d78b9da907dc14725abd 2023-03-24 21:54:12,956 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679694852956"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694852956"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694852956"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694852956"}]},"ts":"1679694852956"} 2023-03-24 21:54:12,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 4467112a96b2d78b9da907dc14725abd 2023-03-24 21:54:12,958 INFO [StoreOpener-4467112a96b2d78b9da907dc14725abd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4467112a96b2d78b9da907dc14725abd 2023-03-24 21:54:12,960 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=30 2023-03-24 21:54:12,960 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=30, state=SUCCESS; OpenRegionProcedure 3080bc8c83bb721d4a0167e2a8e9a57c, server=jenkins-hbase9.apache.org,41807,1679694819103 in 274 msec 2023-03-24 21:54:12,960 DEBUG [StoreOpener-4467112a96b2d78b9da907dc14725abd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/4467112a96b2d78b9da907dc14725abd/family1 2023-03-24 21:54:12,960 DEBUG [StoreOpener-4467112a96b2d78b9da907dc14725abd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/4467112a96b2d78b9da907dc14725abd/family1 2023-03-24 21:54:12,961 INFO [StoreOpener-4467112a96b2d78b9da907dc14725abd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4467112a96b2d78b9da907dc14725abd columnFamilyName family1 2023-03-24 21:54:12,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3080bc8c83bb721d4a0167e2a8e9a57c, ASSIGN in 434 msec 2023-03-24 21:54:12,962 INFO [StoreOpener-4467112a96b2d78b9da907dc14725abd-1] regionserver.HStore(310): Store=4467112a96b2d78b9da907dc14725abd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:12,963 INFO [StoreOpener-4467112a96b2d78b9da907dc14725abd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4467112a96b2d78b9da907dc14725abd 2023-03-24 21:54:12,965 DEBUG [StoreOpener-4467112a96b2d78b9da907dc14725abd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/4467112a96b2d78b9da907dc14725abd/family2 2023-03-24 21:54:12,965 DEBUG [StoreOpener-4467112a96b2d78b9da907dc14725abd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/4467112a96b2d78b9da907dc14725abd/family2 2023-03-24 21:54:12,966 INFO [StoreOpener-4467112a96b2d78b9da907dc14725abd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4467112a96b2d78b9da907dc14725abd columnFamilyName family2 2023-03-24 21:54:12,967 INFO [StoreOpener-4467112a96b2d78b9da907dc14725abd-1] regionserver.HStore(310): Store=4467112a96b2d78b9da907dc14725abd/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:12,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/4467112a96b2d78b9da907dc14725abd 2023-03-24 21:54:12,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/4467112a96b2d78b9da907dc14725abd 2023-03-24 21:54:12,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:54:12,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 4467112a96b2d78b9da907dc14725abd 2023-03-24 21:54:12,980 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/4467112a96b2d78b9da907dc14725abd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:54:12,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 4467112a96b2d78b9da907dc14725abd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75297472, jitterRate=0.12201976776123047}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:54:12,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 4467112a96b2d78b9da907dc14725abd: 2023-03-24 21:54:12,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd., pid=31, masterSystemTime=1679694852837 2023-03-24 21:54:12,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:54:12,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:54:12,985 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4467112a96b2d78b9da907dc14725abd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:12,985 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1679694852985"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694852985"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694852985"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694852985"}]},"ts":"1679694852985"} 2023-03-24 21:54:12,992 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=29 2023-03-24 21:54:12,992 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=29, state=SUCCESS; OpenRegionProcedure 4467112a96b2d78b9da907dc14725abd, server=jenkins-hbase9.apache.org,41807,1679694819103 in 304 msec 2023-03-24 21:54:12,996 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=27 2023-03-24 21:54:12,996 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4467112a96b2d78b9da907dc14725abd, ASSIGN in 466 msec 2023-03-24 21:54:12,997 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:54:12,997 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694852997"}]},"ts":"1679694852997"} 2023-03-24 21:54:12,999 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-03-24 21:54:13,001 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:54:13,005 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 921 msec 2023-03-24 21:54:13,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-24 21:54:13,188 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/.tmp/family1/085e449cda5f4bf68338a78ca877ab73 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/085e449cda5f4bf68338a78ca877ab73 2023-03-24 21:54:13,188 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-03-24 21:54:13,206 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 11f0b79484a112e4405e43efbfd3e887/family1 of 11f0b79484a112e4405e43efbfd3e887 into 085e449cda5f4bf68338a78ca877ab73(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-24 21:54:13,206 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:54:13,206 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1679694851944; duration=1sec 2023-03-24 21:54:13,207 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:13,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:13,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:13,214 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-24 21:54:13,214 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:13,214 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-24 21:54:13,214 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-24 21:54:13,214 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:13,214 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:13,216 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:13,216 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:13,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:13,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:13,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:13,217 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:13,217 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:13,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:13,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:13,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:13,217 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:13,219 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:13,219 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-24 21:54:13,219 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:13,219 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-24 21:54:13,231 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:54:13,232 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. started... 2023-03-24 21:54:13,244 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a7163a0a63017bfa630fabe660e05be6: 2023-03-24 21:54:13,244 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:54:13,244 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:54:13,244 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:13,244 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:54:13,244 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:54:13,244 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. started... 2023-03-24 21:54:13,244 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:54:13,244 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. started... 2023-03-24 21:54:13,245 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 3080bc8c83bb721d4a0167e2a8e9a57c 2/2 column families, dataSize=48 B heapSize=632 B 2023-03-24 21:54:13,250 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 4467112a96b2d78b9da907dc14725abd: 2023-03-24 21:54:13,250 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:54:13,250 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:54:13,279 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/c9cbed81f3d643e79bc7a99a59c4852d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/a314f860ef5e409281a75df2baac19cd, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/09f392ed75544439a4553328cf1504ba_SeqId_47_] to archive 2023-03-24 21:54:13,287 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:13,290 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/c9cbed81f3d643e79bc7a99a59c4852d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/c9cbed81f3d643e79bc7a99a59c4852d 2023-03-24 21:54:13,292 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/a314f860ef5e409281a75df2baac19cd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/a314f860ef5e409281a75df2baac19cd 2023-03-24 21:54:13,294 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/09f392ed75544439a4553328cf1504ba_SeqId_47_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/family1/09f392ed75544439a4553328cf1504ba_SeqId_47_ 2023-03-24 21:54:13,726 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303241f0e54568de74970868269c5bdba2102 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303241f0e54568de74970868269c5bdba2102 2023-03-24 21:54:14,134 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/.tmp/family1/434c95918c3a4eac90e183c499166088 2023-03-24 21:54:14,144 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/.tmp/family1/434c95918c3a4eac90e183c499166088 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/434c95918c3a4eac90e183c499166088 2023-03-24 21:54:14,152 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/434c95918c3a4eac90e183c499166088, entries=1, sequenceid=5, filesize=4.9 K 2023-03-24 21:54:14,154 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 3080bc8c83bb721d4a0167e2a8e9a57c in 909ms, sequenceid=5, compaction requested=false 2023-03-24 21:54:14,154 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-03-24 21:54:14,155 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 3080bc8c83bb721d4a0167e2a8e9a57c: 2023-03-24 21:54:14,155 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:54:14,155 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:54:14,155 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:54:14,155 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,155 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-24 21:54:14,155 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-24 21:54:14,157 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,157 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:14,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:14,157 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,157 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:14,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:14,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:14,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:14,159 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-24 21:54:14,159 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@90f4cd4[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:14,159 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-24 21:54:14,159 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,166 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,167 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-24 21:54:14,167 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-24 21:54:14,167 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:14,167 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,169 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:14,170 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-24 21:54:14,170 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,170 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:14,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:14,171 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-24 21:54:14,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:14,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:14,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:14,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:14,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:14,173 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-24 21:54:14,173 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-24 21:54:14,173 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:14,173 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:14,173 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,174 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:14,175 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,175 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:14,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,175 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:14,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:14,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:14,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:14,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:14,177 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:14,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,178 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:14,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,186 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,186 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,186 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,186 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:14,186 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:14,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-24 21:54:14,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:14,186 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,187 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:14,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:14,187 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:14,187 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:14,186 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:14,187 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:14,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:14,189 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,287 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:14,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:14,294 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:14,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:14,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,296 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-24 21:54:14,296 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:14,296 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-24 21:54:14,296 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-24 21:54:14,297 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,297 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,299 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:14,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:14,299 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:14,300 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,300 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:14,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,301 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:14,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,302 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,303 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-03-24 21:54:14,303 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,303 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-24 21:54:14,303 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:14,303 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-24 21:54:14,306 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:54:14,306 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. started... 2023-03-24 21:54:14,306 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:54:14,306 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. started... 2023-03-24 21:54:14,307 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:54:14,307 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:14,307 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 4467112a96b2d78b9da907dc14725abd: 2023-03-24 21:54:14,307 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:54:14,307 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:54:14,307 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. started... 2023-03-24 21:54:14,307 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:54:14,308 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 3080bc8c83bb721d4a0167e2a8e9a57c 2/2 column families, dataSize=90 B heapSize=752 B 2023-03-24 21:54:14,311 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a7163a0a63017bfa630fabe660e05be6: 2023-03-24 21:54:14,311 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:54:14,311 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:54:14,336 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303243d8dee6a68c645469255a9bf2ac5392d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303243d8dee6a68c645469255a9bf2ac5392d 2023-03-24 21:54:14,367 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/.tmp/family1/559802890f5d47ca9db232dc61ffb726 2023-03-24 21:54:14,375 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/.tmp/family1/559802890f5d47ca9db232dc61ffb726 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/559802890f5d47ca9db232dc61ffb726 2023-03-24 21:54:14,381 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/559802890f5d47ca9db232dc61ffb726, entries=2, sequenceid=9, filesize=5.0 K 2023-03-24 21:54:14,382 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 3080bc8c83bb721d4a0167e2a8e9a57c in 74ms, sequenceid=9, compaction requested=false 2023-03-24 21:54:14,383 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 3080bc8c83bb721d4a0167e2a8e9a57c: 2023-03-24 21:54:14,383 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:54:14,383 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:54:14,383 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:54:14,383 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,383 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-24 21:54:14,383 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-24 21:54:14,385 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,385 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:14,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:14,385 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,385 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:14,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:14,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:14,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:14,389 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-24 21:54:14,389 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b4b6591[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:14,389 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-24 21:54:14,389 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,391 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,392 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,392 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:14,394 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-24 21:54:14,394 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-24 21:54:14,394 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:14,396 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-24 21:54:14,396 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:14,396 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-24 21:54:14,396 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:14,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-24 21:54:14,399 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-24 21:54:14,399 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:14,399 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:14,399 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,399 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:14,403 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,403 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:14,403 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:14,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:14,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:14,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:14,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:14,405 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,407 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:14,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:14,411 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-24 21:54:14,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:14,410 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:14,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:14,411 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:14,411 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,411 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,411 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:14,411 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:14,411 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:14,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:14,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:14,512 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:14,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:14,522 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-03-24 21:54:14,525 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-24 21:54:14,526 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-24 21:54:14,527 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695454527 2023-03-24 21:54:14,528 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:14,536 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-24 21:54:14,536 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-03-24 21:54:14,536 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-24 21:54:14,536 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x7b7148d9 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:54:14,557 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6aa47ddf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:54:14,557 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230324 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-24 21:54:14,662 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303240ca4501badc04371bcfd6fd2ca912f9d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303240ca4501badc04371bcfd6fd2ca912f9d 2023-03-24 21:54:14,664 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230324 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-24 21:54:14,686 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:54:14,688 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:53444, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:54:14,702 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:54:14,705 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:41566, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:54:14,755 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230324/family1/3e203283759c45ab849243242f752829 first=Optional[k0] last=Optional[k1] 2023-03-24 21:54:14,757 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230324/family1/3e203283759c45ab849243242f752829}] 2023-03-24 21:54:14,765 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230324/family1/3e203283759c45ab849243242f752829 for inclusion in 3080bc8c83bb721d4a0167e2a8e9a57c/family1 2023-03-24 21:54:14,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-03-24 21:54:14,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-24 21:54:14,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 3080bc8c83bb721d4a0167e2a8e9a57c: 2023-03-24 21:54:14,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230324/family1/3e203283759c45ab849243242f752829 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testScannerOnBulkLoadRefHFiles__6t5u8c8p4g3pkrp0v9dohk7roonk28rf575foqk2gv39hchdhv3qivanvgd1jtmj/family1/3e203283759c45ab849243242f752829 2023-03-24 21:54:14,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testScannerOnBulkLoadRefHFiles__6t5u8c8p4g3pkrp0v9dohk7roonk28rf575foqk2gv39hchdhv3qivanvgd1jtmj/family1/3e203283759c45ab849243242f752829 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/f263a3fcff6d47208ca1add10d26e3cc_SeqId_12_ 2023-03-24 21:54:14,775 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testScannerOnBulkLoadRefHFiles__6t5u8c8p4g3pkrp0v9dohk7roonk28rf575foqk2gv39hchdhv3qivanvgd1jtmj/family1/3e203283759c45ab849243242f752829 into 3080bc8c83bb721d4a0167e2a8e9a57c/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/f263a3fcff6d47208ca1add10d26e3cc_SeqId_12_ - updating store file list. 2023-03-24 21:54:14,781 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/f263a3fcff6d47208ca1add10d26e3cc_SeqId_12_ into 3080bc8c83bb721d4a0167e2a8e9a57c/family1 2023-03-24 21:54:14,781 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testScannerOnBulkLoadRefHFiles__6t5u8c8p4g3pkrp0v9dohk7roonk28rf575foqk2gv39hchdhv3qivanvgd1jtmj/family1/3e203283759c45ab849243242f752829 into 3080bc8c83bb721d4a0167e2a8e9a57c/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/f263a3fcff6d47208ca1add10d26e3cc_SeqId_12_) 2023-03-24 21:54:14,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testScannerOnBulkLoadRefHFiles__6t5u8c8p4g3pkrp0v9dohk7roonk28rf575foqk2gv39hchdhv3qivanvgd1jtmj/family1/3e203283759c45ab849243242f752829 2023-03-24 21:54:14,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-24 21:54:14,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-24 21:54:14,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(1912): 3080bc8c83bb721d4a0167e2a8e9a57c/family1 is initiating minor compaction (all files) 2023-03-24 21:54:14,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@32a35bb8; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:14,783 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3080bc8c83bb721d4a0167e2a8e9a57c/family1 in testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:54:14,783 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/434c95918c3a4eac90e183c499166088, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/559802890f5d47ca9db232dc61ffb726, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/f263a3fcff6d47208ca1add10d26e3cc_SeqId_12_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/.tmp, totalSize=15.1 K 2023-03-24 21:54:14,787 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 434c95918c3a4eac90e183c499166088, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694853189 2023-03-24 21:54:14,788 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230324 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-24 21:54:14,788 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 559802890f5d47ca9db232dc61ffb726, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694853189 2023-03-24 21:54:14,788 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting f263a3fcff6d47208ca1add10d26e3cc_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694853189 2023-03-24 21:54:14,789 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:14,793 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303241f0e54568de74970868269c5bdba2102 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303241f0e54568de74970868269c5bdba2102 2023-03-24 21:54:14,795 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303243d8dee6a68c645469255a9bf2ac5392d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303243d8dee6a68c645469255a9bf2ac5392d 2023-03-24 21:54:14,795 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-03-24 21:54:14,795 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230324' 2023-03-24 21:54:14,799 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230324 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-24 21:54:14,806 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-24 21:54:14,806 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:14,814 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-24 21:54:14,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 290 msec 2023-03-24 21:54:14,822 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3080bc8c83bb721d4a0167e2a8e9a57c#family1#compaction#38 average throughput is 0.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:54:14,850 INFO [Listener at localhost/39907] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=272 (was 266) - Thread LEAK? -, OpenFileDescriptor=635 (was 623) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=341 (was 341), ProcessCount=169 (was 169), AvailableMemoryMB=8278 (was 8372) 2023-03-24 21:54:14,877 INFO [Listener at localhost/39907] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=274, OpenFileDescriptor=639, MaxFileDescriptor=60000, SystemLoadAverage=341, ProcessCount=169, AvailableMemoryMB=8277 2023-03-24 21:54:14,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-24 21:54:14,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:14,884 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:54:14,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-03-24 21:54:14,885 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:54:14,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-24 21:54:14,886 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe 2023-03-24 21:54:14,887 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe empty. 2023-03-24 21:54:14,888 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe 2023-03-24 21:54:14,888 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-03-24 21:54:14,907 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-24 21:54:14,909 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => c1b745780636112ed0b2ce550374a1fe, NAME => 'testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:54:14,926 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:14,926 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing c1b745780636112ed0b2ce550374a1fe, disabling compactions & flushes 2023-03-24 21:54:14,926 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:14,926 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:14,926 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. after waiting 0 ms 2023-03-24 21:54:14,926 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:14,926 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:14,926 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:14,929 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:54:14,930 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679694854930"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694854930"}]},"ts":"1679694854930"} 2023-03-24 21:54:14,932 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-24 21:54:14,933 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:54:14,933 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694854933"}]},"ts":"1679694854933"} 2023-03-24 21:54:14,935 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-24 21:54:14,940 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=c1b745780636112ed0b2ce550374a1fe, ASSIGN}] 2023-03-24 21:54:14,942 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=c1b745780636112ed0b2ce550374a1fe, ASSIGN 2023-03-24 21:54:14,943 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=c1b745780636112ed0b2ce550374a1fe, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:54:14,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-24 21:54:15,095 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=c1b745780636112ed0b2ce550374a1fe, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:15,095 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679694855095"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694855095"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694855095"}]},"ts":"1679694855095"} 2023-03-24 21:54:15,098 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure c1b745780636112ed0b2ce550374a1fe, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:15,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-24 21:54:15,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:15,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c1b745780636112ed0b2ce550374a1fe, NAME => 'testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:54:15,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy c1b745780636112ed0b2ce550374a1fe 2023-03-24 21:54:15,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:15,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for c1b745780636112ed0b2ce550374a1fe 2023-03-24 21:54:15,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for c1b745780636112ed0b2ce550374a1fe 2023-03-24 21:54:15,257 INFO [StoreOpener-c1b745780636112ed0b2ce550374a1fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c1b745780636112ed0b2ce550374a1fe 2023-03-24 21:54:15,263 DEBUG [StoreOpener-c1b745780636112ed0b2ce550374a1fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1 2023-03-24 21:54:15,263 DEBUG [StoreOpener-c1b745780636112ed0b2ce550374a1fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1 2023-03-24 21:54:15,263 INFO [StoreOpener-c1b745780636112ed0b2ce550374a1fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c1b745780636112ed0b2ce550374a1fe columnFamilyName family1 2023-03-24 21:54:15,266 INFO [StoreOpener-c1b745780636112ed0b2ce550374a1fe-1] regionserver.HStore(310): Store=c1b745780636112ed0b2ce550374a1fe/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:15,268 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe 2023-03-24 21:54:15,268 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe 2023-03-24 21:54:15,273 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for c1b745780636112ed0b2ce550374a1fe 2023-03-24 21:54:15,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:54:15,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened c1b745780636112ed0b2ce550374a1fe; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59333303, jitterRate=-0.11586488783359528}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:54:15,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:15,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe., pid=37, masterSystemTime=1679694855251 2023-03-24 21:54:15,285 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=c1b745780636112ed0b2ce550374a1fe, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:15,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:15,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:15,285 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679694855284"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694855284"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694855284"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694855284"}]},"ts":"1679694855284"} 2023-03-24 21:54:15,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-03-24 21:54:15,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure c1b745780636112ed0b2ce550374a1fe, server=jenkins-hbase9.apache.org,41807,1679694819103 in 188 msec 2023-03-24 21:54:15,295 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-03-24 21:54:15,295 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=c1b745780636112ed0b2ce550374a1fe, ASSIGN in 352 msec 2023-03-24 21:54:15,295 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:54:15,296 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694855295"}]},"ts":"1679694855295"} 2023-03-24 21:54:15,297 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-24 21:54:15,306 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:54:15,307 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 426 msec 2023-03-24 21:54:15,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-24 21:54:15,494 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-03-24 21:54:15,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:15,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:15,535 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:15,536 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:15,536 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:15,536 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:15,536 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:15,536 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:15,538 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:15,538 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:15,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:15,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:15,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:15,539 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:15,539 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:15,539 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:15,540 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:15,540 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:15,540 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:15,540 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:15,540 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:15,543 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:15,543 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:15,543 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:15,543 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. started... 2023-03-24 21:54:15,544 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1b745780636112ed0b2ce550374a1fe 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-24 21:54:15,687 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/.tmp/family1/958ffc33cade42708ecd124bb228f8cb as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/958ffc33cade42708ecd124bb228f8cb 2023-03-24 21:54:15,695 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3080bc8c83bb721d4a0167e2a8e9a57c/family1 of 3080bc8c83bb721d4a0167e2a8e9a57c into 958ffc33cade42708ecd124bb228f8cb(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:54:15,695 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3080bc8c83bb721d4a0167e2a8e9a57c: 2023-03-24 21:54:15,696 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1679694854783; duration=0sec 2023-03-24 21:54:15,696 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:15,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/434c95918c3a4eac90e183c499166088, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/559802890f5d47ca9db232dc61ffb726, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/f263a3fcff6d47208ca1add10d26e3cc_SeqId_12_] to archive 2023-03-24 21:54:15,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:15,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/434c95918c3a4eac90e183c499166088 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/434c95918c3a4eac90e183c499166088 2023-03-24 21:54:15,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/559802890f5d47ca9db232dc61ffb726 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/559802890f5d47ca9db232dc61ffb726 2023-03-24 21:54:15,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/f263a3fcff6d47208ca1add10d26e3cc_SeqId_12_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/family1/f263a3fcff6d47208ca1add10d26e3cc_SeqId_12_ 2023-03-24 21:54:16,010 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130efe111e3fc664e3aaecf9350f9911d78 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130efe111e3fc664e3aaecf9350f9911d78 2023-03-24 21:54:16,018 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/f435913a2bf74f689af046dc65413625 2023-03-24 21:54:16,036 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/f435913a2bf74f689af046dc65413625 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/f435913a2bf74f689af046dc65413625 2023-03-24 21:54:16,056 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/f435913a2bf74f689af046dc65413625, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-24 21:54:16,058 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for c1b745780636112ed0b2ce550374a1fe in 514ms, sequenceid=5, compaction requested=false 2023-03-24 21:54:16,058 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:16,105 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:16,105 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:54:16,105 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2d3f3388 to 127.0.0.1:54680 2023-03-24 21:54:16,105 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:16,106 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:54:16,106 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5ed3108e to 127.0.0.1:54680 2023-03-24 21:54:16,106 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:16,107 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:54:16,107 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x139197bb to 127.0.0.1:54680 2023-03-24 21:54:16,107 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:16,107 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:54:16,107 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x705970c9 to 127.0.0.1:54680 2023-03-24 21:54:16,107 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:16,108 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:54:16,105 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:16,108 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:16,108 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:16,108 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7b7148d9 to 127.0.0.1:54680 2023-03-24 21:54:16,109 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:16,108 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,109 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:16,109 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:16,216 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,216 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:16,217 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,217 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16c1116a[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:16,218 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:16,218 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,220 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,220 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,220 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:16,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,220 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:16,220 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:16,220 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:16,222 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,222 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:16,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:16,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:16,222 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:16,222 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:16,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:16,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:16,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:16,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:16,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:16,226 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:16,226 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:16,226 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:16,226 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,226 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:16,229 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,229 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:16,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,229 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:16,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:16,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:16,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:16,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:16,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,234 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:16,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,235 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:16,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,240 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,240 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,240 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,240 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:16,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:16,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:16,240 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,240 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,240 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,240 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,240 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:16,240 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:16,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:16,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:16,240 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:16,240 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:16,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:16,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:16,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:16,341 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:16,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:16,346 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:16,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:16,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,347 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:16,348 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:16,348 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:16,348 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:16,348 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,348 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,349 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:16,349 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:16,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:16,350 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,350 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:16,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,350 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,351 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,352 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:16,352 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,352 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:16,352 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:16,352 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:16,352 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:16,352 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:16,352 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:16,352 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. started... 2023-03-24 21:54:16,352 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1b745780636112ed0b2ce550374a1fe 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-24 21:54:16,367 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511282231a7ea065a4ea18dc4d71246c49dc3 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511282231a7ea065a4ea18dc4d71246c49dc3 2023-03-24 21:54:16,379 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/2f64d01927b94b20b90e5e665991723f 2023-03-24 21:54:16,384 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/2f64d01927b94b20b90e5e665991723f as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/2f64d01927b94b20b90e5e665991723f 2023-03-24 21:54:16,389 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/2f64d01927b94b20b90e5e665991723f, entries=1, sequenceid=9, filesize=5.0 K 2023-03-24 21:54:16,389 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for c1b745780636112ed0b2ce550374a1fe in 37ms, sequenceid=9, compaction requested=false 2023-03-24 21:54:16,390 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:16,390 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:16,390 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:16,390 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:16,390 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,390 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:16,390 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:16,391 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,391 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:16,392 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:16,392 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:16,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:16,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:16,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@877ef48[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:16,394 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:16,394 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,398 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,399 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:16,399 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:16,399 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:16,399 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,399 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:16,400 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,400 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:16,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:16,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:16,400 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:16,400 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:16,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:16,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:16,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:16,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:16,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:16,403 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:16,403 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:16,403 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:16,403 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,403 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:16,404 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,404 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:16,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:16,404 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:16,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:16,405 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:16,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:16,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:16,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,406 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,410 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:16,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:16,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:16,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:16,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:16,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:16,410 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:16,410 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:16,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:16,410 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:16,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:16,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:16,510 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:16,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:16,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:16,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:16,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,517 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:16,517 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:16,517 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:16,517 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:16,518 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,518 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,519 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:16,519 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:16,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:16,520 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:16,520 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:16,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:16,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,520 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,521 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:16,521 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:16,521 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:16,521 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:16,521 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:16,521 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:16,521 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:16,521 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:16,521 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. started... 2023-03-24 21:54:16,521 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1b745780636112ed0b2ce550374a1fe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:16,942 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201d36efff2b3ee4da0817b9c4496f9e1d5 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201d36efff2b3ee4da0817b9c4496f9e1d5 2023-03-24 21:54:17,349 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/620bac9657a244cdae5b21a37e8abaf6 2023-03-24 21:54:17,354 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/620bac9657a244cdae5b21a37e8abaf6 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/620bac9657a244cdae5b21a37e8abaf6 2023-03-24 21:54:17,358 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/620bac9657a244cdae5b21a37e8abaf6, entries=1, sequenceid=13, filesize=4.9 K 2023-03-24 21:54:17,359 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1b745780636112ed0b2ce550374a1fe in 838ms, sequenceid=13, compaction requested=true 2023-03-24 21:54:17,360 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:17,360 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:17,360 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:17,360 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:17,360 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,360 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:17,360 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:17,362 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,362 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:17,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:17,362 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,362 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:17,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:17,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:17,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:17,364 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:17,364 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b6d7b7e[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:17,364 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:17,364 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,365 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,365 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:17,365 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:17,365 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:17,365 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,365 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:17,367 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:17,367 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,367 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:17,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,367 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:17,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:17,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:17,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:17,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:17,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:17,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:17,370 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:17,370 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:17,370 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:17,370 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:17,370 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,370 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:17,372 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,372 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:17,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:17,372 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:17,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,373 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:17,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:17,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:17,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:17,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,374 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:17,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,376 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,376 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,376 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,376 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,376 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:17,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:17,376 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:17,376 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:17,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:17,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:17,377 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:17,376 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:17,377 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:17,376 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,376 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:17,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:17,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:17,477 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:17,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:17,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:17,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:17,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,485 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:17,486 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:17,486 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:17,486 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:17,486 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,486 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,488 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:17,488 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:17,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:17,489 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,489 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:17,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,490 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:17,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,491 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,491 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:17,491 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,491 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:17,491 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:17,491 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:17,503 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:17,503 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:17,515 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:17,515 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. started... 2023-03-24 21:54:17,515 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1b745780636112ed0b2ce550374a1fe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:17,938 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c37a70612211419395aecd7826f9dcdd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c37a70612211419395aecd7826f9dcdd 2023-03-24 21:54:17,944 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/e9fc6556bae14441beeecc2639d44927 2023-03-24 21:54:17,949 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/e9fc6556bae14441beeecc2639d44927 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/e9fc6556bae14441beeecc2639d44927 2023-03-24 21:54:17,954 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/e9fc6556bae14441beeecc2639d44927, entries=1, sequenceid=17, filesize=4.9 K 2023-03-24 21:54:17,955 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1b745780636112ed0b2ce550374a1fe in 440ms, sequenceid=17, compaction requested=true 2023-03-24 21:54:17,955 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:17,956 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:17,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:17,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:17,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:17,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:17,957 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,957 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:17,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:17,958 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:17,958 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:17,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:17,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:17,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:17,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b5aee6f[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:17,960 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:17,960 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,962 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,962 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:17,962 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,963 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:17,962 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:17,963 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:17,964 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,964 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:17,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:17,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:17,964 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:17,964 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:17,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:17,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:17,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:17,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:17,975 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:17,975 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:17,975 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:17,975 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:17,975 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,975 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:17,978 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,978 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:17,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:17,978 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:17,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:17,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:17,978 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:17,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:17,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,981 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:17,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,991 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,991 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,991 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,991 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:17,991 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:17,991 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:17,991 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:17,991 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:17,991 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:17,991 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,991 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:17,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:17,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:17,992 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:17,991 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,992 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:17,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:17,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:18,092 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:18,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:18,101 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:18,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:18,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,103 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:18,103 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:18,103 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:18,103 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:18,104 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,104 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,109 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,109 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:18,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:18,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:18,110 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,110 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:18,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,111 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:18,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,111 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,119 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:18,119 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,119 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:18,119 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:18,119 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:18,119 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:18,119 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:18,119 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:18,120 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. started... 2023-03-24 21:54:18,120 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1b745780636112ed0b2ce550374a1fe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:18,167 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122869e8b33e76154a0d97b4f76a2aca419c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122869e8b33e76154a0d97b4f76a2aca419c 2023-03-24 21:54:18,386 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-24 21:54:18,576 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/ed7b2a062d1a452c8899b1b64543f9b9 2023-03-24 21:54:18,584 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/ed7b2a062d1a452c8899b1b64543f9b9 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/ed7b2a062d1a452c8899b1b64543f9b9 2023-03-24 21:54:18,589 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/ed7b2a062d1a452c8899b1b64543f9b9, entries=1, sequenceid=21, filesize=4.9 K 2023-03-24 21:54:18,590 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1b745780636112ed0b2ce550374a1fe in 470ms, sequenceid=21, compaction requested=true 2023-03-24 21:54:18,590 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:18,590 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:18,590 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:18,590 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:18,590 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,590 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:18,590 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:18,592 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,592 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:18,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:18,592 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,592 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:18,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:18,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:18,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:18,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:18,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@350ef02f[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:18,594 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:18,594 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,595 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,595 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:18,595 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:18,595 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:18,595 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,595 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:18,601 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,601 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:18,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:18,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:18,601 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:18,601 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:18,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:18,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:18,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:18,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:18,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:18,604 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:18,604 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:18,604 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:18,604 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,604 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:18,606 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,606 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:18,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,607 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:18,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:18,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:18,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,612 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,613 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,616 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,616 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,616 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:18,616 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:18,616 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,616 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:18,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:18,616 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:18,616 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,616 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,616 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:18,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:18,616 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:18,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:18,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:18,616 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:18,717 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:18,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:18,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:18,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:18,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,724 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:18,725 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:18,725 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:18,725 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:18,725 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,725 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,727 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,727 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:18,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:18,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:18,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:18,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:18,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,729 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,735 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:18,735 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,735 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:18,735 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:18,735 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:18,735 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:18,735 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:18,736 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:18,736 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. started... 2023-03-24 21:54:18,736 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1b745780636112ed0b2ce550374a1fe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:18,763 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512315d8ef48da8da4645ad1d808596212368 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512315d8ef48da8da4645ad1d808596212368 2023-03-24 21:54:18,773 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/92d35915870c47ecad6db05e251b1542 2023-03-24 21:54:18,780 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/92d35915870c47ecad6db05e251b1542 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/92d35915870c47ecad6db05e251b1542 2023-03-24 21:54:18,786 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/92d35915870c47ecad6db05e251b1542, entries=1, sequenceid=25, filesize=4.9 K 2023-03-24 21:54:18,787 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1b745780636112ed0b2ce550374a1fe in 51ms, sequenceid=25, compaction requested=true 2023-03-24 21:54:18,787 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:18,787 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:18,787 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:18,787 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:18,787 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,787 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:18,787 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:18,790 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,790 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:18,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:18,790 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,790 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:18,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:18,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:18,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:18,792 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:18,792 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2344469[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:18,792 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:18,792 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,793 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,793 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:18,793 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:18,793 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:18,794 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,794 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:18,795 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,795 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:18,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:18,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:18,795 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:18,795 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:18,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:18,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:18,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:18,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:18,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:18,798 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:18,798 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:18,798 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:18,798 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,798 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:18,800 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,800 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,800 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:18,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:18,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:18,800 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:18,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:18,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:18,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:18,801 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:18,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,809 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:18,809 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:18,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:18,809 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:18,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:18,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:18,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:18,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:18,809 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:18,809 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,810 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:18,810 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,810 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:18,810 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,810 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,810 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,810 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,910 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:18,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:18,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:18,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:18,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,918 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:18,918 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:18,918 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:18,918 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:18,919 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,919 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,920 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:18,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:18,920 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:18,920 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:18,921 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:18,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:18,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:18,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:18,922 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:18,922 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:18,922 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:18,922 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:18,922 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:18,922 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:18,922 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. started... 2023-03-24 21:54:18,922 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1b745780636112ed0b2ce550374a1fe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:19,338 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010121656f4b7cef44b1a951e16bf41de7a2 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010121656f4b7cef44b1a951e16bf41de7a2 2023-03-24 21:54:19,344 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/3617407edd124d79878ee5bfdb3e3337 2023-03-24 21:54:19,349 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/3617407edd124d79878ee5bfdb3e3337 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/3617407edd124d79878ee5bfdb3e3337 2023-03-24 21:54:19,363 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/3617407edd124d79878ee5bfdb3e3337, entries=1, sequenceid=29, filesize=4.9 K 2023-03-24 21:54:19,364 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1b745780636112ed0b2ce550374a1fe in 442ms, sequenceid=29, compaction requested=true 2023-03-24 21:54:19,364 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:19,364 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:19,365 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:19,365 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:19,365 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,365 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:19,365 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:19,366 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,366 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:19,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:19,367 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,367 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:19,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:19,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:19,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:19,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:19,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@121269da[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:19,375 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:19,375 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,377 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,377 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:19,377 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,377 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:19,377 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:19,377 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:19,381 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,381 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:19,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:19,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:19,381 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:19,381 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:19,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:19,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:19,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:19,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:19,385 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:19,385 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:19,385 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:19,385 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:19,385 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,385 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:19,387 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,387 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:19,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,387 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:19,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:19,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:19,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:19,388 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:19,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:19,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,389 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:19,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,396 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,396 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:19,396 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:19,396 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:19,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:19,396 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:19,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:19,397 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:19,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:19,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:19,397 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,397 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:19,397 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:19,397 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:19,397 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,397 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,397 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,497 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:19,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:19,503 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:19,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:19,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,505 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:19,505 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:19,505 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:19,505 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:19,505 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,505 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,507 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,507 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:19,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:19,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:19,507 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,507 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:19,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:19,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,509 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:19,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,509 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:19,509 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:19,509 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:19,509 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:19,510 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:19,510 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:19,510 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. started... 2023-03-24 21:54:19,510 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1b745780636112ed0b2ce550374a1fe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:19,534 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010327cb9ea1a07540448af6dc8bcda26fba to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010327cb9ea1a07540448af6dc8bcda26fba 2023-03-24 21:54:19,540 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/4854850788e64267ae0520cf5f9ebf96 2023-03-24 21:54:19,544 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/4854850788e64267ae0520cf5f9ebf96 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/4854850788e64267ae0520cf5f9ebf96 2023-03-24 21:54:19,551 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/4854850788e64267ae0520cf5f9ebf96, entries=1, sequenceid=33, filesize=4.9 K 2023-03-24 21:54:19,552 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1b745780636112ed0b2ce550374a1fe in 42ms, sequenceid=33, compaction requested=true 2023-03-24 21:54:19,553 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:19,553 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:19,553 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:19,553 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:19,553 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,553 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:19,553 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:19,554 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,554 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:19,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:19,555 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,555 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:19,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:19,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:19,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:19,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:19,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e47d461[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:19,559 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:19,559 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,560 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,560 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:19,561 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:19,561 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:19,561 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,561 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:19,563 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:19,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:19,564 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:19,564 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:19,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:19,564 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:19,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:19,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:19,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:19,566 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:19,566 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:19,566 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:19,566 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:19,566 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,566 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:19,568 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,568 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,568 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:19,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:19,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:19,569 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:19,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:19,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:19,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:19,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,570 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:19,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,573 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,573 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,573 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:19,573 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:19,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:19,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:19,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:19,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:19,575 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,575 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,575 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,575 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:19,575 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:19,575 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:19,576 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:19,575 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:19,575 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,578 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,580 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:19,676 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:19,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:19,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:19,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:19,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,693 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:19,693 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:19,693 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:19,693 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:19,693 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,694 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,695 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:19,695 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:19,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:19,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,696 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:19,696 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:19,696 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:19,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,697 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,697 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:19,697 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:19,697 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:19,697 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:19,697 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:19,700 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:19,700 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:19,700 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:19,700 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. started... 2023-03-24 21:54:19,701 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1b745780636112ed0b2ce550374a1fe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:19,730 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090787e965c3d8d24d008c5e61d173455fca to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015090787e965c3d8d24d008c5e61d173455fca 2023-03-24 21:54:20,144 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/d08fedfc47b644f2ba6c6aa6b8cf4d56 2023-03-24 21:54:20,152 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/d08fedfc47b644f2ba6c6aa6b8cf4d56 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/d08fedfc47b644f2ba6c6aa6b8cf4d56 2023-03-24 21:54:20,161 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/d08fedfc47b644f2ba6c6aa6b8cf4d56, entries=1, sequenceid=37, filesize=4.9 K 2023-03-24 21:54:20,163 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1b745780636112ed0b2ce550374a1fe in 462ms, sequenceid=37, compaction requested=true 2023-03-24 21:54:20,163 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:20,163 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:20,163 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:20,163 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:20,163 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,163 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:20,163 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:20,166 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,166 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:20,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:20,167 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,167 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:20,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:20,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:20,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:20,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:20,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c0ade0e[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:20,169 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:20,169 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,171 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,171 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:20,171 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:20,171 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:20,172 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,172 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:20,173 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:20,173 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:20,173 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:20,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:20,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:20,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:20,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:20,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:20,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:20,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:20,176 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:20,176 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:20,176 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:20,176 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,176 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:20,178 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,178 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:20,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:20,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,178 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:20,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:20,179 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:20,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:20,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:20,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,181 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:20,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,184 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:20,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:20,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:20,184 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:20,185 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:20,185 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:20,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:20,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:20,185 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,185 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,185 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,185 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,185 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,185 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,185 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:20,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:20,186 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:20,186 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:20,286 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:20,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:20,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:20,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:20,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,293 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:20,294 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:20,294 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-24 21:54:20,294 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-24 21:54:20,294 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,294 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,296 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:20,296 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:20,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:20,296 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,296 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:20,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,297 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:20,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,297 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,297 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-24 21:54:20,297 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,298 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-24 21:54:20,298 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:20,298 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-24 21:54:20,298 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:20,298 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:20,298 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:20,298 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. started... 2023-03-24 21:54:20,298 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1b745780636112ed0b2ce550374a1fe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:20,322 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112000a1a38d01174f118ece7e47f9491891 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112000a1a38d01174f118ece7e47f9491891 2023-03-24 21:54:20,728 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/068e4dc790974bb7a85e407d137ac620 2023-03-24 21:54:20,733 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/068e4dc790974bb7a85e407d137ac620 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/068e4dc790974bb7a85e407d137ac620 2023-03-24 21:54:20,738 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/068e4dc790974bb7a85e407d137ac620, entries=1, sequenceid=41, filesize=4.9 K 2023-03-24 21:54:20,739 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1b745780636112ed0b2ce550374a1fe in 441ms, sequenceid=41, compaction requested=true 2023-03-24 21:54:20,739 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:20,739 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:20,739 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:20,740 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:20,740 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,740 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-24 21:54:20,740 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-24 21:54:20,741 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,741 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:20,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:20,742 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,742 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:20,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:20,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:20,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:20,743 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-24 21:54:20,743 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c3dd0ea[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:20,743 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-24 21:54:20,743 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,745 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,745 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-24 21:54:20,745 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-24 21:54:20,745 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:20,745 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,745 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:20,749 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-24 21:54:20,749 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,749 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:20,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:20,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:20,749 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-24 21:54:20,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:20,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:20,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:20,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:20,751 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:54:20,751 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-24 21:54:20,751 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:20,752 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:20,752 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,752 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:20,753 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:20,753 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:20,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,753 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:20,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:20,754 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:20,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:20,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:20,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,755 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:20,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,760 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,760 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,760 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,760 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:20,760 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:20,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-24 21:54:20,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:20,760 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:20,760 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:20,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:20,760 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:20,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-24 21:54:20,761 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:20,761 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:20,760 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,761 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:20,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:20,861 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:20,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:20,863 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-03-24 21:54:20,865 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-24 21:54:20,866 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-24 21:54:20,867 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695460867 2023-03-24 21:54:20,868 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:20,869 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-24 21:54:20,870 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-24 21:54:20,870 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-24 21:54:20,870 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x3c0f5d0f to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:54:20,876 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@750c95d4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:54:20,876 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:21,337 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103df4c3c5245b54ad09b8d7f4ae36197f9 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103df4c3c5245b54ad09b8d7f4ae36197f9 2023-03-24 21:54:21,338 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-24 21:54:21,352 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:54:21,354 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:46878, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:54:21,361 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:54:21,362 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:53066, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:54:21,376 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/efc7938daf7b4b3aa9e164ce26da31a4 first=Optional[r3] last=Optional[r6] 2023-03-24 21:54:21,377 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/efc7938daf7b4b3aa9e164ce26da31a4}] 2023-03-24 21:54:21,381 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/efc7938daf7b4b3aa9e164ce26da31a4 for inclusion in c1b745780636112ed0b2ce550374a1fe/family1 2023-03-24 21:54:21,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-24 21:54:21,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:54:21,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:21,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/efc7938daf7b4b3aa9e164ce26da31a4 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__7k4up2jeajam5fnl65mu44rcto1dv5hpm5pil1b9kq71q2t5tqj2o7sj0mb29gl7/family1/efc7938daf7b4b3aa9e164ce26da31a4 2023-03-24 21:54:21,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__7k4up2jeajam5fnl65mu44rcto1dv5hpm5pil1b9kq71q2t5tqj2o7sj0mb29gl7/family1/efc7938daf7b4b3aa9e164ce26da31a4 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/886c672d660b4b9793f1e4c500bf9cb4_SeqId_44_ 2023-03-24 21:54:21,388 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__7k4up2jeajam5fnl65mu44rcto1dv5hpm5pil1b9kq71q2t5tqj2o7sj0mb29gl7/family1/efc7938daf7b4b3aa9e164ce26da31a4 into c1b745780636112ed0b2ce550374a1fe/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/886c672d660b4b9793f1e4c500bf9cb4_SeqId_44_ - updating store file list. 2023-03-24 21:54:21,395 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/886c672d660b4b9793f1e4c500bf9cb4_SeqId_44_ into c1b745780636112ed0b2ce550374a1fe/family1 2023-03-24 21:54:21,395 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__7k4up2jeajam5fnl65mu44rcto1dv5hpm5pil1b9kq71q2t5tqj2o7sj0mb29gl7/family1/efc7938daf7b4b3aa9e164ce26da31a4 into c1b745780636112ed0b2ce550374a1fe/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/886c672d660b4b9793f1e4c500bf9cb4_SeqId_44_) 2023-03-24 21:54:21,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__7k4up2jeajam5fnl65mu44rcto1dv5hpm5pil1b9kq71q2t5tqj2o7sj0mb29gl7/family1/efc7938daf7b4b3aa9e164ce26da31a4 2023-03-24 21:54:21,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-24 21:54:21,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-24 21:54:21,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(1912): c1b745780636112ed0b2ce550374a1fe/family1 is initiating minor compaction 2023-03-24 21:54:21,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@472b337; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:21,399 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c1b745780636112ed0b2ce550374a1fe/family1 in testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:54:21,399 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/2f64d01927b94b20b90e5e665991723f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/620bac9657a244cdae5b21a37e8abaf6, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/e9fc6556bae14441beeecc2639d44927, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/ed7b2a062d1a452c8899b1b64543f9b9, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/92d35915870c47ecad6db05e251b1542, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/3617407edd124d79878ee5bfdb3e3337, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/4854850788e64267ae0520cf5f9ebf96, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/d08fedfc47b644f2ba6c6aa6b8cf4d56, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/068e4dc790974bb7a85e407d137ac620, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/886c672d660b4b9793f1e4c500bf9cb4_SeqId_44_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp, totalSize=50.0 K 2023-03-24 21:54:21,399 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 2f64d01927b94b20b90e5e665991723f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-24 21:54:21,399 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 620bac9657a244cdae5b21a37e8abaf6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-24 21:54:21,399 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting e9fc6556bae14441beeecc2639d44927, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-24 21:54:21,399 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting ed7b2a062d1a452c8899b1b64543f9b9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-24 21:54:21,400 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 92d35915870c47ecad6db05e251b1542, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-24 21:54:21,400 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 3617407edd124d79878ee5bfdb3e3337, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-24 21:54:21,400 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 4854850788e64267ae0520cf5f9ebf96, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-24 21:54:21,400 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting d08fedfc47b644f2ba6c6aa6b8cf4d56, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-24 21:54:21,400 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 068e4dc790974bb7a85e407d137ac620, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-24 21:54:21,400 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 886c672d660b4b9793f1e4c500bf9cb4_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-24 21:54:21,403 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-24 21:54:21,404 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:21,407 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122869e8b33e76154a0d97b4f76a2aca419c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122869e8b33e76154a0d97b4f76a2aca419c 2023-03-24 21:54:21,409 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512315d8ef48da8da4645ad1d808596212368 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512315d8ef48da8da4645ad1d808596212368 2023-03-24 21:54:21,410 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010121656f4b7cef44b1a951e16bf41de7a2 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010121656f4b7cef44b1a951e16bf41de7a2 2023-03-24 21:54:21,411 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010327cb9ea1a07540448af6dc8bcda26fba to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010327cb9ea1a07540448af6dc8bcda26fba 2023-03-24 21:54:21,411 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-24 21:54:21,412 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-24 21:54:21,412 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:21,412 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:21,412 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:21,412 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:21,412 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:21,412 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:21,412 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:21,413 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:21,439 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): c1b745780636112ed0b2ce550374a1fe#family1#compaction#49 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:54:21,482 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205bbda04f61a5c4cee9aa054ddca42e9d5 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205bbda04f61a5c4cee9aa054ddca42e9d5 2023-03-24 21:54:21,484 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-24 21:54:21,527 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/05417f6398444930aa8012d19136ed12 first=Optional[r00] last=Optional[r2] 2023-03-24 21:54:21,528 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/05417f6398444930aa8012d19136ed12}] 2023-03-24 21:54:21,539 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/05417f6398444930aa8012d19136ed12 for inclusion in c1b745780636112ed0b2ce550374a1fe/family1 2023-03-24 21:54:21,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-24 21:54:21,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:54:21,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:21,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/05417f6398444930aa8012d19136ed12 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__iqemfd28blhgv3uq7de6jepl8i78k0t9atckbis3l1g147a4t0g3gq1n08j7h05p/family1/05417f6398444930aa8012d19136ed12 2023-03-24 21:54:21,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__iqemfd28blhgv3uq7de6jepl8i78k0t9atckbis3l1g147a4t0g3gq1n08j7h05p/family1/05417f6398444930aa8012d19136ed12 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/a6c930b3631e41d4b98567cf065c2f99_SeqId_46_ 2023-03-24 21:54:21,551 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__iqemfd28blhgv3uq7de6jepl8i78k0t9atckbis3l1g147a4t0g3gq1n08j7h05p/family1/05417f6398444930aa8012d19136ed12 into c1b745780636112ed0b2ce550374a1fe/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/a6c930b3631e41d4b98567cf065c2f99_SeqId_46_ - updating store file list. 2023-03-24 21:54:21,558 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/a6c930b3631e41d4b98567cf065c2f99_SeqId_46_ into c1b745780636112ed0b2ce550374a1fe/family1 2023-03-24 21:54:21,558 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__iqemfd28blhgv3uq7de6jepl8i78k0t9atckbis3l1g147a4t0g3gq1n08j7h05p/family1/05417f6398444930aa8012d19136ed12 into c1b745780636112ed0b2ce550374a1fe/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/a6c930b3631e41d4b98567cf065c2f99_SeqId_46_) 2023-03-24 21:54:21,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__iqemfd28blhgv3uq7de6jepl8i78k0t9atckbis3l1g147a4t0g3gq1n08j7h05p/family1/05417f6398444930aa8012d19136ed12 2023-03-24 21:54:21,562 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-24 21:54:21,563 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:21,566 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130efe111e3fc664e3aaecf9350f9911d78 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130efe111e3fc664e3aaecf9350f9911d78 2023-03-24 21:54:21,569 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201d36efff2b3ee4da0817b9c4496f9e1d5 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201d36efff2b3ee4da0817b9c4496f9e1d5 2023-03-24 21:54:21,570 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c37a70612211419395aecd7826f9dcdd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c37a70612211419395aecd7826f9dcdd 2023-03-24 21:54:21,570 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-03-24 21:54:21,571 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-24 21:54:21,571 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-24 21:54:21,571 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-24 21:54:21,571 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:21,576 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-24 21:54:21,578 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 712 msec 2023-03-24 21:54:21,717 INFO [Listener at localhost/39907] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=270 (was 274), OpenFileDescriptor=631 (was 639), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=361 (was 341) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=8164 (was 8277) 2023-03-24 21:54:21,734 INFO [Listener at localhost/39907] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=270, OpenFileDescriptor=631, MaxFileDescriptor=60000, SystemLoadAverage=388, ProcessCount=169, AvailableMemoryMB=8164 2023-03-24 21:54:21,738 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-24 21:54:21,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:21,741 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:54:21,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-03-24 21:54:21,742 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:54:21,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-24 21:54:21,744 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:21,745 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df empty. 2023-03-24 21:54:21,745 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:21,745 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-03-24 21:54:21,758 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-24 21:54:21,760 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 471fea1dd43c43f9175a965d62ffe2df, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:54:21,771 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:21,771 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing 471fea1dd43c43f9175a965d62ffe2df, disabling compactions & flushes 2023-03-24 21:54:21,771 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:21,771 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:21,771 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. after waiting 0 ms 2023-03-24 21:54:21,771 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:21,771 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:21,771 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:21,774 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:54:21,775 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679694861775"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694861775"}]},"ts":"1679694861775"} 2023-03-24 21:54:21,776 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-24 21:54:21,777 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:54:21,777 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694861777"}]},"ts":"1679694861777"} 2023-03-24 21:54:21,778 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-03-24 21:54:21,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=471fea1dd43c43f9175a965d62ffe2df, ASSIGN}] 2023-03-24 21:54:21,783 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=471fea1dd43c43f9175a965d62ffe2df, ASSIGN 2023-03-24 21:54:21,783 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=471fea1dd43c43f9175a965d62ffe2df, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:54:21,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-24 21:54:21,935 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=471fea1dd43c43f9175a965d62ffe2df, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:21,935 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679694861935"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694861935"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694861935"}]},"ts":"1679694861935"} 2023-03-24 21:54:21,937 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 471fea1dd43c43f9175a965d62ffe2df, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:22,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-24 21:54:22,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:22,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 471fea1dd43c43f9175a965d62ffe2df, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:54:22,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:22,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:22,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:22,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:22,094 INFO [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:22,095 DEBUG [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1 2023-03-24 21:54:22,095 DEBUG [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1 2023-03-24 21:54:22,095 INFO [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 471fea1dd43c43f9175a965d62ffe2df columnFamilyName family1 2023-03-24 21:54:22,097 INFO [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] regionserver.HStore(310): Store=471fea1dd43c43f9175a965d62ffe2df/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:22,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:22,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:22,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:22,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:54:22,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 471fea1dd43c43f9175a965d62ffe2df; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70977206, jitterRate=0.05764278769493103}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:54:22,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:22,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df., pid=41, masterSystemTime=1679694862088 2023-03-24 21:54:22,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:22,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:22,108 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=471fea1dd43c43f9175a965d62ffe2df, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,108 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679694862108"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694862108"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694862108"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694862108"}]},"ts":"1679694862108"} 2023-03-24 21:54:22,111 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-03-24 21:54:22,112 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 471fea1dd43c43f9175a965d62ffe2df, server=jenkins-hbase9.apache.org,41807,1679694819103 in 173 msec 2023-03-24 21:54:22,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-03-24 21:54:22,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=471fea1dd43c43f9175a965d62ffe2df, ASSIGN in 331 msec 2023-03-24 21:54:22,114 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:54:22,114 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694862114"}]},"ts":"1679694862114"} 2023-03-24 21:54:22,115 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-03-24 21:54:22,117 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:54:22,118 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 379 msec 2023-03-24 21:54:22,260 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/.tmp/family1/672cce25107f481391c22dc7a19b7e58 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/672cce25107f481391c22dc7a19b7e58 2023-03-24 21:54:22,266 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in c1b745780636112ed0b2ce550374a1fe/family1 of c1b745780636112ed0b2ce550374a1fe into 672cce25107f481391c22dc7a19b7e58(size=6.6 K), total size for store is 337.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:54:22,266 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:54:22,266 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1679694861399; duration=0sec 2023-03-24 21:54:22,266 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:22,273 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/2f64d01927b94b20b90e5e665991723f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/620bac9657a244cdae5b21a37e8abaf6, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/e9fc6556bae14441beeecc2639d44927, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/ed7b2a062d1a452c8899b1b64543f9b9, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/92d35915870c47ecad6db05e251b1542, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/3617407edd124d79878ee5bfdb3e3337, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/4854850788e64267ae0520cf5f9ebf96, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/d08fedfc47b644f2ba6c6aa6b8cf4d56, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/068e4dc790974bb7a85e407d137ac620, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/886c672d660b4b9793f1e4c500bf9cb4_SeqId_44_] to archive 2023-03-24 21:54:22,274 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:22,276 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/2f64d01927b94b20b90e5e665991723f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/2f64d01927b94b20b90e5e665991723f 2023-03-24 21:54:22,277 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/620bac9657a244cdae5b21a37e8abaf6 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/620bac9657a244cdae5b21a37e8abaf6 2023-03-24 21:54:22,278 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/e9fc6556bae14441beeecc2639d44927 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/e9fc6556bae14441beeecc2639d44927 2023-03-24 21:54:22,280 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/ed7b2a062d1a452c8899b1b64543f9b9 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/ed7b2a062d1a452c8899b1b64543f9b9 2023-03-24 21:54:22,281 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/92d35915870c47ecad6db05e251b1542 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/92d35915870c47ecad6db05e251b1542 2023-03-24 21:54:22,282 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/3617407edd124d79878ee5bfdb3e3337 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/3617407edd124d79878ee5bfdb3e3337 2023-03-24 21:54:22,283 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/4854850788e64267ae0520cf5f9ebf96 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/4854850788e64267ae0520cf5f9ebf96 2023-03-24 21:54:22,285 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/d08fedfc47b644f2ba6c6aa6b8cf4d56 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/d08fedfc47b644f2ba6c6aa6b8cf4d56 2023-03-24 21:54:22,286 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/068e4dc790974bb7a85e407d137ac620 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/068e4dc790974bb7a85e407d137ac620 2023-03-24 21:54:22,287 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/886c672d660b4b9793f1e4c500bf9cb4_SeqId_44_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/family1/886c672d660b4b9793f1e4c500bf9cb4_SeqId_44_ 2023-03-24 21:54:22,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-24 21:54:22,346 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-03-24 21:54:22,372 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:22,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,378 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:22,378 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:22,378 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-24 21:54:22,378 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-24 21:54:22,378 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,379 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,380 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:22,380 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:22,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:22,380 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,380 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:22,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,381 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:22,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,381 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,381 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,382 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-24 21:54:22,382 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:22,382 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-24 21:54:22,382 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:22,382 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:22,382 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:22,382 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. started... 2023-03-24 21:54:22,383 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 471fea1dd43c43f9175a965d62ffe2df 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-24 21:54:22,431 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113029e27a75363b4356b8284fe63f19bf60 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113029e27a75363b4356b8284fe63f19bf60 2023-03-24 21:54:22,438 WARN [DataStreamer for file /user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/49477c14f22243248fee65d1ebe4cf62] hdfs.DataStreamer(982): Caught exception java.lang.InterruptedException at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1257) at java.lang.Thread.join(Thread.java:1331) at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:980) at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:630) at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:807) 2023-03-24 21:54:22,438 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/49477c14f22243248fee65d1ebe4cf62 2023-03-24 21:54:22,445 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/49477c14f22243248fee65d1ebe4cf62 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/49477c14f22243248fee65d1ebe4cf62 2023-03-24 21:54:22,454 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/49477c14f22243248fee65d1ebe4cf62, entries=1000, sequenceid=5, filesize=191.4 K 2023-03-24 21:54:22,455 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 471fea1dd43c43f9175a965d62ffe2df in 71ms, sequenceid=5, compaction requested=false 2023-03-24 21:54:22,455 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:22,455 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:22,455 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:22,455 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:22,456 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:22,456 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,456 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-24 21:54:22,456 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-24 21:54:22,459 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,459 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:22,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:22,459 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:22,459 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:22,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:22,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:22,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-24 21:54:22,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@de0240e[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:22,461 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-24 21:54:22,461 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,464 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,464 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-24 21:54:22,464 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-24 21:54:22,464 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:22,464 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,464 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:22,466 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,466 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-24 21:54:22,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:22,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:22,466 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:22,466 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-24 21:54:22,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:22,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:22,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:22,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:22,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-24 21:54:22,469 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-24 21:54:22,469 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:22,469 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:22,469 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,469 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:22,474 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,474 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,474 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:22,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:22,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:22,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:22,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:22,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:22,475 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,476 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:22,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,478 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,478 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,478 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:22,478 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:22,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:22,478 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:22,478 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:22,478 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:22,479 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:22,479 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:22,479 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:22,479 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,479 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:22,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:22,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:22,479 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-24 21:54:22,579 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:22,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:22,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:22,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:22,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,587 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:22,587 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:22,587 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-24 21:54:22,587 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-24 21:54:22,589 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,589 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,591 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:22,591 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:22,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:22,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,592 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,592 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:22,592 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:22,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,592 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,593 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-24 21:54:22,593 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,593 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-24 21:54:22,593 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:22,593 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-24 21:54:22,594 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:22,594 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:22,594 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:22,594 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. started... 2023-03-24 21:54:22,594 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 471fea1dd43c43f9175a965d62ffe2df 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-24 21:54:22,617 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112823b8421a91934693b99051453ff3db07 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112823b8421a91934693b99051453ff3db07 2023-03-24 21:54:22,624 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/c61c8409236648899d9ead2102891a00 2023-03-24 21:54:22,630 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/c61c8409236648899d9ead2102891a00 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/c61c8409236648899d9ead2102891a00 2023-03-24 21:54:22,635 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/c61c8409236648899d9ead2102891a00, entries=1, sequenceid=9, filesize=5.0 K 2023-03-24 21:54:22,636 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 471fea1dd43c43f9175a965d62ffe2df in 42ms, sequenceid=9, compaction requested=false 2023-03-24 21:54:22,636 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:22,636 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:22,636 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:22,636 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:22,636 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,636 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-24 21:54:22,636 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-24 21:54:22,638 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,638 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:22,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:22,638 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,638 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:22,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:22,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:22,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:22,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-24 21:54:22,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d541d0f[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:22,640 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-24 21:54:22,640 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,641 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,642 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-24 21:54:22,642 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-24 21:54:22,642 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:22,642 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,642 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:22,643 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-24 21:54:22,643 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:22,643 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,643 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-24 21:54:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:22,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:22,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:22,648 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-24 21:54:22,648 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-24 21:54:22,648 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:22,648 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:22,648 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,648 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:22,651 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,651 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:22,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:22,652 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:22,652 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:22,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,654 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:22,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,665 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,665 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,665 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:22,665 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:22,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:22,665 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:22,665 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:22,665 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:22,665 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,665 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,665 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,665 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,665 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,665 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:22,665 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:22,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:22,666 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-24 21:54:22,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:22,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:22,766 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:22,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:22,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:22,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:22,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,784 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:22,785 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:22,785 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-24 21:54:22,785 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-24 21:54:22,785 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,785 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,787 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:22,787 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:22,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:22,787 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:22,787 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:22,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,788 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:22,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,788 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,789 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-24 21:54:22,789 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:22,789 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-24 21:54:22,789 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:22,789 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-24 21:54:22,789 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:22,789 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:22,789 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:22,789 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. started... 2023-03-24 21:54:22,789 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 471fea1dd43c43f9175a965d62ffe2df 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:23,214 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120160caa4cc91fb453f8fcd1fc1133bcd2d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120160caa4cc91fb453f8fcd1fc1133bcd2d 2023-03-24 21:54:23,620 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/b394b4bee3de4b3aa86e16c4f0eff813 2023-03-24 21:54:23,626 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/b394b4bee3de4b3aa86e16c4f0eff813 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/b394b4bee3de4b3aa86e16c4f0eff813 2023-03-24 21:54:23,631 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/b394b4bee3de4b3aa86e16c4f0eff813, entries=1, sequenceid=13, filesize=5.0 K 2023-03-24 21:54:23,632 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 471fea1dd43c43f9175a965d62ffe2df in 843ms, sequenceid=13, compaction requested=true 2023-03-24 21:54:23,632 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:23,632 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:23,632 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:23,632 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:23,632 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,632 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-24 21:54:23,632 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-24 21:54:23,637 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:23,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:23,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:23,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:23,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,638 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:23,639 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,639 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:23,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-24 21:54:23,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27544205[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:23,639 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-24 21:54:23,639 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,640 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,641 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-24 21:54:23,641 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-24 21:54:23,641 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:23,641 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,641 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:23,642 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:23,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:23,642 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-24 21:54:23,642 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:23,642 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-24 21:54:23,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:23,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:23,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:23,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:23,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-24 21:54:23,645 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-24 21:54:23,645 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:23,645 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:23,645 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,646 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:23,647 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,647 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,647 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:23,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:23,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:23,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:23,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:23,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:23,648 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:23,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,650 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:23,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,654 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:23,654 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,654 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:23,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:23,654 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,654 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:23,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:23,654 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-24 21:54:23,654 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:23,654 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,654 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,654 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,654 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:23,655 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:23,655 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:23,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:23,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:23,755 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:23,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:23,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:23,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:23,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,768 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:23,768 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:23,768 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-24 21:54:23,768 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-24 21:54:23,768 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,768 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,769 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:23,770 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:23,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:23,770 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:23,770 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:23,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,770 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:23,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,771 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,771 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-24 21:54:23,771 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:23,771 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-24 21:54:23,771 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:23,771 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-24 21:54:23,771 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:23,772 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:23,772 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:23,772 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. started... 2023-03-24 21:54:23,772 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 471fea1dd43c43f9175a965d62ffe2df 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:24,192 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512053026b596ea034bc8a265893dc2e973b6 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512053026b596ea034bc8a265893dc2e973b6 2023-03-24 21:54:24,205 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/6f75dbb144304d41a80fed7073738d83 2023-03-24 21:54:24,211 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/6f75dbb144304d41a80fed7073738d83 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/6f75dbb144304d41a80fed7073738d83 2023-03-24 21:54:24,216 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/6f75dbb144304d41a80fed7073738d83, entries=1, sequenceid=17, filesize=5.0 K 2023-03-24 21:54:24,217 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 471fea1dd43c43f9175a965d62ffe2df in 445ms, sequenceid=17, compaction requested=true 2023-03-24 21:54:24,217 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:24,217 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:24,217 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:24,217 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:24,217 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,217 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-24 21:54:24,217 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-24 21:54:24,219 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,219 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:24,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:24,219 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,219 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:24,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:24,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:24,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:24,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-24 21:54:24,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@458c6c85[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:24,221 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-24 21:54:24,221 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,222 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,222 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-24 21:54:24,222 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-24 21:54:24,222 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:24,222 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,222 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:24,225 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,225 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-24 21:54:24,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:24,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:24,225 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:24,225 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-24 21:54:24,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:24,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:24,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:24,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:24,227 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-24 21:54:24,227 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-24 21:54:24,227 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:24,227 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:24,227 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,227 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:24,229 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,229 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,229 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:24,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:24,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:24,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:24,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:24,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:24,230 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:24,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,231 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:24,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,233 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,234 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,234 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:24,234 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:24,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:24,234 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:24,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:24,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:24,234 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-24 21:54:24,234 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:24,234 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,234 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,234 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,234 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,234 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:24,234 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:24,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:24,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:24,334 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:24,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:24,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:24,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:24,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,341 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:24,341 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:24,341 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-24 21:54:24,341 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-24 21:54:24,341 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,342 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,343 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:24,343 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:24,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:24,343 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,343 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:24,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:24,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,344 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,345 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-24 21:54:24,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,345 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-24 21:54:24,345 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:24,345 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-24 21:54:24,345 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:24,345 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:24,345 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:24,345 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. started... 2023-03-24 21:54:24,345 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 471fea1dd43c43f9175a965d62ffe2df 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:24,762 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512284008ef82bf634fadb65b7e59f090daba to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512284008ef82bf634fadb65b7e59f090daba 2023-03-24 21:54:24,771 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/58e831cb5c7041ed83147fded377260e 2023-03-24 21:54:24,776 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/58e831cb5c7041ed83147fded377260e as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/58e831cb5c7041ed83147fded377260e 2023-03-24 21:54:24,780 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/58e831cb5c7041ed83147fded377260e, entries=1, sequenceid=21, filesize=5.0 K 2023-03-24 21:54:24,781 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 471fea1dd43c43f9175a965d62ffe2df in 436ms, sequenceid=21, compaction requested=true 2023-03-24 21:54:24,781 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:24,781 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:24,781 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:24,781 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:24,781 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,781 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-24 21:54:24,781 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-24 21:54:24,787 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,787 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:24,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:24,787 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,787 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:24,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:24,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:24,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:24,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-24 21:54:24,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4940e06e[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:24,788 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-24 21:54:24,788 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,789 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,790 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-24 21:54:24,790 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-24 21:54:24,790 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:24,790 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,790 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:24,791 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-24 21:54:24,791 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,791 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:24,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,791 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-24 21:54:24,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:24,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:24,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:24,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:24,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:24,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:24,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-24 21:54:24,794 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-24 21:54:24,794 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:24,794 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:24,794 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,794 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:24,796 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,796 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:24,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:24,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,796 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:24,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:24,796 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:24,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:24,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:24,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,801 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:24,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,805 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,805 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,805 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,805 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,805 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,805 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:24,805 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:24,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:24,805 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:24,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:24,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:24,805 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:24,806 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-24 21:54:24,806 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:24,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:24,806 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:24,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:24,906 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:24,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:24,911 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:24,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:24,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,913 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:24,913 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:24,913 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-24 21:54:24,913 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-24 21:54:24,914 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,914 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,915 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:24,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:24,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:24,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,916 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:24,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,916 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,917 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-24 21:54:24,917 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:24,917 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-24 21:54:24,917 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:24,917 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-24 21:54:24,917 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,917 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:24,917 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:24,919 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:24,919 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:24,919 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:24,919 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. started... 2023-03-24 21:54:24,919 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 471fea1dd43c43f9175a965d62ffe2df 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:25,333 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512315f2da4bcfe1945d69975e00457e9ea9e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512315f2da4bcfe1945d69975e00457e9ea9e 2023-03-24 21:54:25,360 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/aef234fcda574e40b5d7f0eaef8f4790 2023-03-24 21:54:25,367 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/aef234fcda574e40b5d7f0eaef8f4790 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/aef234fcda574e40b5d7f0eaef8f4790 2023-03-24 21:54:25,372 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/aef234fcda574e40b5d7f0eaef8f4790, entries=1, sequenceid=25, filesize=5.0 K 2023-03-24 21:54:25,373 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 471fea1dd43c43f9175a965d62ffe2df in 454ms, sequenceid=25, compaction requested=true 2023-03-24 21:54:25,374 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:25,374 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:25,374 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:25,374 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:25,374 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,374 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-24 21:54:25,374 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-24 21:54:25,376 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:25,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:25,376 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:25,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:25,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,378 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,378 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:25,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-24 21:54:25,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3401202c[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:25,379 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-24 21:54:25,379 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,380 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,380 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-24 21:54:25,380 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-24 21:54:25,380 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:25,381 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,381 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:25,383 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:25,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:25,383 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-24 21:54:25,383 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:25,383 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-24 21:54:25,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:25,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:25,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:25,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:25,386 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-24 21:54:25,386 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-24 21:54:25,386 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:25,386 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:25,387 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,387 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:25,388 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,388 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:25,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,389 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:25,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:25,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:25,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:25,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:25,390 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:25,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,392 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:25,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,394 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:25,394 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:25,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:25,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:25,395 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:25,395 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,395 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,395 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:25,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:25,395 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,395 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,395 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:25,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-24 21:54:25,395 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:25,395 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:25,395 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:25,496 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:25,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:25,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:25,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:25,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,515 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:25,515 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:25,515 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-24 21:54:25,515 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-24 21:54:25,516 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,516 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,518 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:25,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:25,518 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:25,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,519 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,519 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:25,519 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:25,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,519 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,528 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-24 21:54:25,528 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,528 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-24 21:54:25,528 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:25,528 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-24 21:54:25,528 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:25,528 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:25,529 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:25,529 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. started... 2023-03-24 21:54:25,529 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 471fea1dd43c43f9175a965d62ffe2df 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:25,573 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101c52686ee3f59474c922a4ac2b52d152c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101c52686ee3f59474c922a4ac2b52d152c 2023-03-24 21:54:25,579 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/03d5ca4982854be3a463ebd8dedeb155 2023-03-24 21:54:25,585 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/03d5ca4982854be3a463ebd8dedeb155 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/03d5ca4982854be3a463ebd8dedeb155 2023-03-24 21:54:25,591 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/03d5ca4982854be3a463ebd8dedeb155, entries=1, sequenceid=29, filesize=5.0 K 2023-03-24 21:54:25,592 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 471fea1dd43c43f9175a965d62ffe2df in 63ms, sequenceid=29, compaction requested=true 2023-03-24 21:54:25,592 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:25,592 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:25,592 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:25,592 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:25,592 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,592 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-24 21:54:25,592 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-24 21:54:25,596 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:25,596 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:25,597 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,597 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:25,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-24 21:54:25,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@314008c3[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:25,598 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-24 21:54:25,598 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,600 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,600 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-24 21:54:25,600 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-24 21:54:25,600 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:25,600 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,600 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:25,601 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-24 21:54:25,601 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:25,601 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-24 21:54:25,601 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:25,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:25,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:25,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:25,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:25,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:25,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-24 21:54:25,605 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-24 21:54:25,605 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:25,605 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:25,605 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,605 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:25,608 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,608 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:25,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,608 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,608 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:25,610 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:25,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,619 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:25,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:25,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:25,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:25,619 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,620 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,620 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,620 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,620 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,620 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,620 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-24 21:54:25,620 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:25,620 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:25,620 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:25,620 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:25,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:25,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:25,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:25,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:25,720 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:25,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:25,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:25,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:25,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,728 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:25,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:25,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-24 21:54:25,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-24 21:54:25,729 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,729 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,730 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:25,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:25,730 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:25,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,731 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:25,731 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:25,731 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:25,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,731 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,734 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-24 21:54:25,734 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:25,734 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-24 21:54:25,734 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:25,734 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-24 21:54:25,734 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:25,734 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:25,734 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:25,734 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. started... 2023-03-24 21:54:25,735 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 471fea1dd43c43f9175a965d62ffe2df 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:26,153 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103db178befa863459bac5db885387c2102 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103db178befa863459bac5db885387c2102 2023-03-24 21:54:26,165 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/f2fdf5a5bc21478f90e1f1ffad932c3c 2023-03-24 21:54:26,176 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/f2fdf5a5bc21478f90e1f1ffad932c3c as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/f2fdf5a5bc21478f90e1f1ffad932c3c 2023-03-24 21:54:26,183 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/f2fdf5a5bc21478f90e1f1ffad932c3c, entries=1, sequenceid=33, filesize=5.0 K 2023-03-24 21:54:26,184 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 471fea1dd43c43f9175a965d62ffe2df in 449ms, sequenceid=33, compaction requested=true 2023-03-24 21:54:26,184 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:26,184 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:26,184 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:26,184 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:26,184 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,184 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-24 21:54:26,184 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-24 21:54:26,187 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,187 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:26,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:26,187 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,187 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:26,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:26,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:26,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:26,191 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-24 21:54:26,191 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@170ea22a[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:26,191 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-24 21:54:26,191 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,192 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,193 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-24 21:54:26,193 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-24 21:54:26,193 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:26,194 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,194 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:26,198 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-24 21:54:26,198 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:26,198 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,198 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-24 21:54:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:26,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:26,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:26,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:26,201 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-24 21:54:26,201 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-24 21:54:26,201 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:26,201 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:26,201 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,201 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:26,203 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,203 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:26,203 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:26,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:26,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:26,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:26,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:26,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,205 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:26,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,207 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:26,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,211 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,211 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,211 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,211 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:26,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:26,211 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:26,211 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,211 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,211 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,211 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:26,211 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:26,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:26,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:26,211 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:26,212 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:26,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:26,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:26,312 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:26,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:26,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:26,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:26,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,319 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:26,319 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:26,319 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-24 21:54:26,319 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-24 21:54:26,319 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,319 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,320 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,320 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:26,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:26,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:26,321 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:26,321 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:26,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,321 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:26,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,321 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,322 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-24 21:54:26,322 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:26,322 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-24 21:54:26,322 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:26,322 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-24 21:54:26,322 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:26,322 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:26,322 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:26,322 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. started... 2023-03-24 21:54:26,323 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 471fea1dd43c43f9175a965d62ffe2df 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:26,739 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907cc51ec30734b44e5866aead87a5a91cd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907cc51ec30734b44e5866aead87a5a91cd 2023-03-24 21:54:27,145 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/9b93ea0f8f374101a8cc07d9d8773845 2023-03-24 21:54:27,150 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/9b93ea0f8f374101a8cc07d9d8773845 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9b93ea0f8f374101a8cc07d9d8773845 2023-03-24 21:54:27,155 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9b93ea0f8f374101a8cc07d9d8773845, entries=1, sequenceid=37, filesize=5.0 K 2023-03-24 21:54:27,156 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 471fea1dd43c43f9175a965d62ffe2df in 833ms, sequenceid=37, compaction requested=true 2023-03-24 21:54:27,156 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:27,156 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:27,156 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:27,156 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:27,156 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,156 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-24 21:54:27,156 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-24 21:54:27,158 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,158 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:27,158 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,158 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:27,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:27,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:27,160 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-24 21:54:27,160 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59836335[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:27,160 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-24 21:54:27,160 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,161 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,161 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-24 21:54:27,161 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-24 21:54:27,161 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:27,161 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,162 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:27,163 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,163 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-24 21:54:27,163 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:27,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:27,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:27,163 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-24 21:54:27,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:27,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:27,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:27,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:27,166 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-24 21:54:27,166 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-24 21:54:27,166 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:27,166 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:27,166 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,166 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:27,169 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,169 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,169 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:27,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:27,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:27,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:27,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:27,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:27,169 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:27,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,171 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:27,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:27,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:27,173 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:27,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:27,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:27,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:27,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:27,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:27,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:27,174 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:27,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,174 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:27,274 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:27,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:27,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:27,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:27,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,281 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:27,281 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:27,281 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-24 21:54:27,281 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-24 21:54:27,282 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,282 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,286 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,286 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:27,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:27,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:27,286 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,286 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:27,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,287 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:27,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,287 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,287 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-24 21:54:27,287 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,288 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-24 21:54:27,288 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:27,288 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-24 21:54:27,288 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:27,288 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:27,288 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:27,288 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. started... 2023-03-24 21:54:27,288 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 471fea1dd43c43f9175a965d62ffe2df 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:27,304 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511200c1bc1d3a0f44988b1161330a20271b0 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511200c1bc1d3a0f44988b1161330a20271b0 2023-03-24 21:54:27,594 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-24 21:54:27,709 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/4f59e690fd754124b758e1c3b497124a 2023-03-24 21:54:27,714 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/4f59e690fd754124b758e1c3b497124a as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/4f59e690fd754124b758e1c3b497124a 2023-03-24 21:54:27,719 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/4f59e690fd754124b758e1c3b497124a, entries=1, sequenceid=41, filesize=5.0 K 2023-03-24 21:54:27,720 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 471fea1dd43c43f9175a965d62ffe2df in 431ms, sequenceid=41, compaction requested=true 2023-03-24 21:54:27,720 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:27,720 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:27,720 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:27,720 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:27,720 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,720 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-24 21:54:27,720 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-24 21:54:27,722 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,722 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:27,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:27,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:27,723 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,723 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:27,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:27,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:27,724 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-24 21:54:27,724 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64be2869[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:27,724 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-24 21:54:27,725 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,726 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,726 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-24 21:54:27,726 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-24 21:54:27,726 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:27,727 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-24 21:54:27,727 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:27,728 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-24 21:54:27,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-24 21:54:27,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:27,728 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-24 21:54:27,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:27,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:27,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,728 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:27,730 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,730 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:27,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,730 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:27,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:27,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:27,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:27,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:27,730 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:27,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,732 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:27,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,735 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,736 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,736 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,736 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:27,736 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:27,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:27,736 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,736 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:27,736 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:27,736 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,736 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-24 21:54:27,736 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:27,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:27,736 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,736 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:27,736 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:27,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:27,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:27,837 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:27,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:27,839 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:27,841 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-24 21:54:27,841 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-24 21:54:27,842 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:27,842 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695467842 2023-03-24 21:54:27,843 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-24 21:54:27,844 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-24 21:54:27,844 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-24 21:54:27,845 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x54f1cde6 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:54:27,853 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@45425099, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:54:27,853 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:28,293 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103e2b1ef7d05b748c89e82eb63327d310d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103e2b1ef7d05b748c89e82eb63327d310d 2023-03-24 21:54:28,295 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-24 21:54:28,310 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:54:28,311 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:46884, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:54:28,318 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:54:28,320 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:53074, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:54:28,330 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/42ae8787cef64527bbc7362e63dafab2 first=Optional[r3] last=Optional[r6] 2023-03-24 21:54:28,331 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/42ae8787cef64527bbc7362e63dafab2}] 2023-03-24 21:54:28,337 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/42ae8787cef64527bbc7362e63dafab2 for inclusion in 471fea1dd43c43f9175a965d62ffe2df/family1 2023-03-24 21:54:28,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-24 21:54:28,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:54:28,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:28,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/42ae8787cef64527bbc7362e63dafab2 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ebkclelgch81602rj5nq6bf24v5r60fr453ne00nrv1hu90vchfmp7rpddjd9vg3/family1/42ae8787cef64527bbc7362e63dafab2 2023-03-24 21:54:28,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ebkclelgch81602rj5nq6bf24v5r60fr453ne00nrv1hu90vchfmp7rpddjd9vg3/family1/42ae8787cef64527bbc7362e63dafab2 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/82be91dc037d4a10a4f845da7984d41f_SeqId_44_ 2023-03-24 21:54:28,344 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ebkclelgch81602rj5nq6bf24v5r60fr453ne00nrv1hu90vchfmp7rpddjd9vg3/family1/42ae8787cef64527bbc7362e63dafab2 into 471fea1dd43c43f9175a965d62ffe2df/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/82be91dc037d4a10a4f845da7984d41f_SeqId_44_ - updating store file list. 2023-03-24 21:54:28,348 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/82be91dc037d4a10a4f845da7984d41f_SeqId_44_ into 471fea1dd43c43f9175a965d62ffe2df/family1 2023-03-24 21:54:28,348 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ebkclelgch81602rj5nq6bf24v5r60fr453ne00nrv1hu90vchfmp7rpddjd9vg3/family1/42ae8787cef64527bbc7362e63dafab2 into 471fea1dd43c43f9175a965d62ffe2df/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/82be91dc037d4a10a4f845da7984d41f_SeqId_44_) 2023-03-24 21:54:28,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ebkclelgch81602rj5nq6bf24v5r60fr453ne00nrv1hu90vchfmp7rpddjd9vg3/family1/42ae8787cef64527bbc7362e63dafab2 2023-03-24 21:54:28,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-24 21:54:28,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-24 21:54:28,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(1912): 471fea1dd43c43f9175a965d62ffe2df/family1 is initiating minor compaction 2023-03-24 21:54:28,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2a7d1222; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:28,353 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 471fea1dd43c43f9175a965d62ffe2df/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:28,353 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/c61c8409236648899d9ead2102891a00, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/b394b4bee3de4b3aa86e16c4f0eff813, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/6f75dbb144304d41a80fed7073738d83, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/58e831cb5c7041ed83147fded377260e, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/aef234fcda574e40b5d7f0eaef8f4790, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/03d5ca4982854be3a463ebd8dedeb155, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/f2fdf5a5bc21478f90e1f1ffad932c3c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9b93ea0f8f374101a8cc07d9d8773845, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/4f59e690fd754124b758e1c3b497124a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/82be91dc037d4a10a4f845da7984d41f_SeqId_44_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp, totalSize=50.4 K 2023-03-24 21:54:28,353 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting c61c8409236648899d9ead2102891a00, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-24 21:54:28,353 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting b394b4bee3de4b3aa86e16c4f0eff813, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-24 21:54:28,353 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 6f75dbb144304d41a80fed7073738d83, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-24 21:54:28,353 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 58e831cb5c7041ed83147fded377260e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-24 21:54:28,353 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting aef234fcda574e40b5d7f0eaef8f4790, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-24 21:54:28,353 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 03d5ca4982854be3a463ebd8dedeb155, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-24 21:54:28,353 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting f2fdf5a5bc21478f90e1f1ffad932c3c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-24 21:54:28,353 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 9b93ea0f8f374101a8cc07d9d8773845, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-24 21:54:28,354 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 4f59e690fd754124b758e1c3b497124a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-24 21:54:28,354 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 82be91dc037d4a10a4f845da7984d41f_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-24 21:54:28,360 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-24 21:54:28,361 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:28,364 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512284008ef82bf634fadb65b7e59f090daba to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512284008ef82bf634fadb65b7e59f090daba 2023-03-24 21:54:28,366 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512315f2da4bcfe1945d69975e00457e9ea9e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512315f2da4bcfe1945d69975e00457e9ea9e 2023-03-24 21:54:28,367 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101c52686ee3f59474c922a4ac2b52d152c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101c52686ee3f59474c922a4ac2b52d152c 2023-03-24 21:54:28,369 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103db178befa863459bac5db885387c2102 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103db178befa863459bac5db885387c2102 2023-03-24 21:54:28,369 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-03-24 21:54:28,369 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-24 21:54:28,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:28,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:28,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:28,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:28,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:28,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:28,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:28,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:28,406 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): 471fea1dd43c43f9175a965d62ffe2df#family1#compaction#60 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:54:29,234 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/e907cf76262a43d99212c868b9334246 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/e907cf76262a43d99212c868b9334246 2023-03-24 21:54:29,241 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 471fea1dd43c43f9175a965d62ffe2df/family1 of 471fea1dd43c43f9175a965d62ffe2df into e907cf76262a43d99212c868b9334246(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:54:29,241 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:29,242 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1679694868352; duration=0sec 2023-03-24 21:54:29,242 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512056b4324d3836840f38b41fd47ec38deae to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512056b4324d3836840f38b41fd47ec38deae 2023-03-24 21:54:29,242 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:29,244 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-24 21:54:29,263 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/c61c8409236648899d9ead2102891a00, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/b394b4bee3de4b3aa86e16c4f0eff813, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/6f75dbb144304d41a80fed7073738d83, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/58e831cb5c7041ed83147fded377260e, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/aef234fcda574e40b5d7f0eaef8f4790, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/03d5ca4982854be3a463ebd8dedeb155, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/f2fdf5a5bc21478f90e1f1ffad932c3c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9b93ea0f8f374101a8cc07d9d8773845, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/4f59e690fd754124b758e1c3b497124a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/82be91dc037d4a10a4f845da7984d41f_SeqId_44_] to archive 2023-03-24 21:54:29,264 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:29,266 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/c61c8409236648899d9ead2102891a00 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/c61c8409236648899d9ead2102891a00 2023-03-24 21:54:29,268 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/b394b4bee3de4b3aa86e16c4f0eff813 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/b394b4bee3de4b3aa86e16c4f0eff813 2023-03-24 21:54:29,269 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/6f75dbb144304d41a80fed7073738d83 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/6f75dbb144304d41a80fed7073738d83 2023-03-24 21:54:29,270 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/58e831cb5c7041ed83147fded377260e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/58e831cb5c7041ed83147fded377260e 2023-03-24 21:54:29,272 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/aef234fcda574e40b5d7f0eaef8f4790 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/aef234fcda574e40b5d7f0eaef8f4790 2023-03-24 21:54:29,274 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/03d5ca4982854be3a463ebd8dedeb155 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/03d5ca4982854be3a463ebd8dedeb155 2023-03-24 21:54:29,277 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/f2fdf5a5bc21478f90e1f1ffad932c3c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/f2fdf5a5bc21478f90e1f1ffad932c3c 2023-03-24 21:54:29,279 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c3539f6f0d6347ffa61092241cc354a5 first=Optional[r00] last=Optional[r2] 2023-03-24 21:54:29,279 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c3539f6f0d6347ffa61092241cc354a5}] 2023-03-24 21:54:29,279 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9b93ea0f8f374101a8cc07d9d8773845 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9b93ea0f8f374101a8cc07d9d8773845 2023-03-24 21:54:29,281 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/4f59e690fd754124b758e1c3b497124a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/4f59e690fd754124b758e1c3b497124a 2023-03-24 21:54:29,282 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c3539f6f0d6347ffa61092241cc354a5 for inclusion in 471fea1dd43c43f9175a965d62ffe2df/family1 2023-03-24 21:54:29,282 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/82be91dc037d4a10a4f845da7984d41f_SeqId_44_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/82be91dc037d4a10a4f845da7984d41f_SeqId_44_ 2023-03-24 21:54:29,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-24 21:54:29,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:54:29,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:29,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c3539f6f0d6347ffa61092241cc354a5 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__8ddgjqae4ie2sqg2utn5n2rrg2qr7diufkjul62u40gce1oresasl2uug0t4bfnj/family1/c3539f6f0d6347ffa61092241cc354a5 2023-03-24 21:54:29,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__8ddgjqae4ie2sqg2utn5n2rrg2qr7diufkjul62u40gce1oresasl2uug0t4bfnj/family1/c3539f6f0d6347ffa61092241cc354a5 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9e11ca6b8d684473910838eec8dc164a_SeqId_47_ 2023-03-24 21:54:29,292 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__8ddgjqae4ie2sqg2utn5n2rrg2qr7diufkjul62u40gce1oresasl2uug0t4bfnj/family1/c3539f6f0d6347ffa61092241cc354a5 into 471fea1dd43c43f9175a965d62ffe2df/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9e11ca6b8d684473910838eec8dc164a_SeqId_47_ - updating store file list. 2023-03-24 21:54:29,296 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9e11ca6b8d684473910838eec8dc164a_SeqId_47_ into 471fea1dd43c43f9175a965d62ffe2df/family1 2023-03-24 21:54:29,296 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__8ddgjqae4ie2sqg2utn5n2rrg2qr7diufkjul62u40gce1oresasl2uug0t4bfnj/family1/c3539f6f0d6347ffa61092241cc354a5 into 471fea1dd43c43f9175a965d62ffe2df/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9e11ca6b8d684473910838eec8dc164a_SeqId_47_) 2023-03-24 21:54:29,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__8ddgjqae4ie2sqg2utn5n2rrg2qr7diufkjul62u40gce1oresasl2uug0t4bfnj/family1/c3539f6f0d6347ffa61092241cc354a5 2023-03-24 21:54:29,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-24 21:54:29,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-24 21:54:29,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(1912): 471fea1dd43c43f9175a965d62ffe2df/family1 is initiating minor compaction (all files) 2023-03-24 21:54:29,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@48b40ece; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:29,298 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 471fea1dd43c43f9175a965d62ffe2df/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:29,298 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/49477c14f22243248fee65d1ebe4cf62, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/e907cf76262a43d99212c868b9334246, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9e11ca6b8d684473910838eec8dc164a_SeqId_47_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp, totalSize=390.1 K 2023-03-24 21:54:29,299 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 49477c14f22243248fee65d1ebe4cf62, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841999 2023-03-24 21:54:29,299 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting e907cf76262a43d99212c868b9334246, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241999 2023-03-24 21:54:29,300 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 9e11ca6b8d684473910838eec8dc164a_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878841999 2023-03-24 21:54:29,304 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-24 21:54:29,306 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:29,309 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113029e27a75363b4356b8284fe63f19bf60 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113029e27a75363b4356b8284fe63f19bf60 2023-03-24 21:54:29,310 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120160caa4cc91fb453f8fcd1fc1133bcd2d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120160caa4cc91fb453f8fcd1fc1133bcd2d 2023-03-24 21:54:29,312 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512053026b596ea034bc8a265893dc2e973b6 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512053026b596ea034bc8a265893dc2e973b6 2023-03-24 21:54:29,312 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-03-24 21:54:29,312 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-24 21:54:29,313 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:29,313 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-24 21:54:29,313 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:29,318 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): 471fea1dd43c43f9175a965d62ffe2df#family1#compaction#61 average throughput is 20.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:54:29,320 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-24 21:54:29,322 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.4800 sec 2023-03-24 21:54:29,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$12(2454): Client=jenkins//172.31.2.10 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-24 21:54:29,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:29,846 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-24 21:54:29,848 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-24 21:54:29,848 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-24 21:54:29,850 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-24 21:54:29,854 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=471fea1dd43c43f9175a965d62ffe2df, REOPEN/MOVE}] 2023-03-24 21:54:29,854 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=471fea1dd43c43f9175a965d62ffe2df, REOPEN/MOVE 2023-03-24 21:54:29,855 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=471fea1dd43c43f9175a965d62ffe2df, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:29,855 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679694869855"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694869855"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694869855"}]},"ts":"1679694869855"} 2023-03-24 21:54:29,857 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 471fea1dd43c43f9175a965d62ffe2df, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:30,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:30,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 471fea1dd43c43f9175a965d62ffe2df, disabling compactions & flushes 2023-03-24 21:54:30,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:30,158 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/.tmp/family1/4e25cb4240d54d40a9ba752b336d3bf1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/4e25cb4240d54d40a9ba752b336d3bf1 2023-03-24 21:54:30,164 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 471fea1dd43c43f9175a965d62ffe2df/family1 of 471fea1dd43c43f9175a965d62ffe2df into 4e25cb4240d54d40a9ba752b336d3bf1(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:54:30,164 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:30,164 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:30,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:30,164 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1679694869298; duration=0sec 2023-03-24 21:54:30,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. after waiting 0 ms 2023-03-24 21:54:30,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:30,164 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:30,166 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/49477c14f22243248fee65d1ebe4cf62, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/e907cf76262a43d99212c868b9334246, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9e11ca6b8d684473910838eec8dc164a_SeqId_47_] to archive 2023-03-24 21:54:30,167 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:30,169 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/49477c14f22243248fee65d1ebe4cf62 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/49477c14f22243248fee65d1ebe4cf62 2023-03-24 21:54:30,170 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/e907cf76262a43d99212c868b9334246 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/e907cf76262a43d99212c868b9334246 2023-03-24 21:54:30,172 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9e11ca6b8d684473910838eec8dc164a_SeqId_47_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/9e11ca6b8d684473910838eec8dc164a_SeqId_47_ 2023-03-24 21:54:30,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-24 21:54:30,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:30,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:30,178 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 471fea1dd43c43f9175a965d62ffe2df to self. 2023-03-24 21:54:30,179 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:30,180 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=471fea1dd43c43f9175a965d62ffe2df, regionState=CLOSED 2023-03-24 21:54:30,180 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679694870180"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694870180"}]},"ts":"1679694870180"} 2023-03-24 21:54:30,182 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-03-24 21:54:30,182 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 471fea1dd43c43f9175a965d62ffe2df, server=jenkins-hbase9.apache.org,41807,1679694819103 in 324 msec 2023-03-24 21:54:30,183 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=471fea1dd43c43f9175a965d62ffe2df, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=true 2023-03-24 21:54:30,334 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=471fea1dd43c43f9175a965d62ffe2df, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:30,334 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679694870334"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694870334"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694870334"}]},"ts":"1679694870334"} 2023-03-24 21:54:30,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 471fea1dd43c43f9175a965d62ffe2df, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:30,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:30,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 471fea1dd43c43f9175a965d62ffe2df, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:54:30,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:30,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:30,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:30,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:30,496 INFO [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:30,497 DEBUG [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1 2023-03-24 21:54:30,497 DEBUG [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1 2023-03-24 21:54:30,498 INFO [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 471fea1dd43c43f9175a965d62ffe2df columnFamilyName family1 2023-03-24 21:54:30,503 DEBUG [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/4e25cb4240d54d40a9ba752b336d3bf1 2023-03-24 21:54:30,503 INFO [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] regionserver.HStore(310): Store=471fea1dd43c43f9175a965d62ffe2df/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:30,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:30,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:30,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:30,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 471fea1dd43c43f9175a965d62ffe2df; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64035823, jitterRate=-0.045791879296302795}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:54:30,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:30,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df., pid=47, masterSystemTime=1679694870491 2023-03-24 21:54:30,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:30,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:30,511 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=471fea1dd43c43f9175a965d62ffe2df, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:30,512 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679694870511"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694870511"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694870511"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694870511"}]},"ts":"1679694870511"} 2023-03-24 21:54:30,514 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-03-24 21:54:30,514 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 471fea1dd43c43f9175a965d62ffe2df, server=jenkins-hbase9.apache.org,41807,1679694819103 in 174 msec 2023-03-24 21:54:30,516 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-03-24 21:54:30,516 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=471fea1dd43c43f9175a965d62ffe2df, REOPEN/MOVE in 661 msec 2023-03-24 21:54:30,518 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-03-24 21:54:30,518 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 666 msec 2023-03-24 21:54:30,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0880 sec 2023-03-24 21:54:30,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-03-24 21:54:30,523 INFO [Listener at localhost/39907] compactions.TestMobCompactor(278): alter status finished 2023-03-24 21:54:30,531 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:30,533 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-24 21:54:30,533 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-24 21:54:30,534 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:30,534 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695470534 2023-03-24 21:54:30,535 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-24 21:54:30,535 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-24 21:54:30,536 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-24 21:54:30,536 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x3f8d6df4 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:54:30,541 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@681c3404, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:54:30,541 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:30,541 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:30,541 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:30,541 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:30,541 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:30,973 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511287089fb23be4347f5bb89a15ce53ca580 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511287089fb23be4347f5bb89a15ce53ca580 2023-03-24 21:54:30,974 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-24 21:54:30,994 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:54:30,995 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:59816, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:54:31,005 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:54:31,008 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:57350, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:54:31,026 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9cdb7db16e50404a8e04151db56fee7c first=Optional[r05] last=Optional[r8] 2023-03-24 21:54:31,027 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=51 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9cdb7db16e50404a8e04151db56fee7c}] 2023-03-24 21:54:31,036 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9cdb7db16e50404a8e04151db56fee7c for inclusion in 471fea1dd43c43f9175a965d62ffe2df/family1 2023-03-24 21:54:31,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-24 21:54:31,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:54:31,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:31,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9cdb7db16e50404a8e04151db56fee7c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g41ihapt2jce15p6bb1kau8rc0n44pr3m7aegkmtt64ci0994gcl2hdveijcq7sq/family1/9cdb7db16e50404a8e04151db56fee7c 2023-03-24 21:54:31,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g41ihapt2jce15p6bb1kau8rc0n44pr3m7aegkmtt64ci0994gcl2hdveijcq7sq/family1/9cdb7db16e50404a8e04151db56fee7c as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/184a78937414431696866e50d6267d6d_SeqId_53_ 2023-03-24 21:54:31,044 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g41ihapt2jce15p6bb1kau8rc0n44pr3m7aegkmtt64ci0994gcl2hdveijcq7sq/family1/9cdb7db16e50404a8e04151db56fee7c into 471fea1dd43c43f9175a965d62ffe2df/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/184a78937414431696866e50d6267d6d_SeqId_53_ - updating store file list. 2023-03-24 21:54:31,048 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/184a78937414431696866e50d6267d6d_SeqId_53_ into 471fea1dd43c43f9175a965d62ffe2df/family1 2023-03-24 21:54:31,048 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g41ihapt2jce15p6bb1kau8rc0n44pr3m7aegkmtt64ci0994gcl2hdveijcq7sq/family1/9cdb7db16e50404a8e04151db56fee7c into 471fea1dd43c43f9175a965d62ffe2df/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/184a78937414431696866e50d6267d6d_SeqId_53_) 2023-03-24 21:54:31,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g41ihapt2jce15p6bb1kau8rc0n44pr3m7aegkmtt64ci0994gcl2hdveijcq7sq/family1/9cdb7db16e50404a8e04151db56fee7c 2023-03-24 21:54:31,050 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-24 21:54:31,050 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:31,052 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511200c1bc1d3a0f44988b1161330a20271b0 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511200c1bc1d3a0f44988b1161330a20271b0 2023-03-24 21:54:31,053 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112823b8421a91934693b99051453ff3db07 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112823b8421a91934693b99051453ff3db07 2023-03-24 21:54:31,053 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-03-24 21:54:31,053 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-24 21:54:31,053 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:31,053 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:31,054 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:31,054 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-24 21:54:31,054 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:31,055 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-24 21:54:31,056 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 523 msec 2023-03-24 21:54:31,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$12(2454): Client=jenkins//172.31.2.10 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-24 21:54:31,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:31,125 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-24 21:54:31,126 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-24 21:54:31,126 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-24 21:54:31,128 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-24 21:54:31,133 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=471fea1dd43c43f9175a965d62ffe2df, REOPEN/MOVE}] 2023-03-24 21:54:31,133 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=471fea1dd43c43f9175a965d62ffe2df, REOPEN/MOVE 2023-03-24 21:54:31,135 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=471fea1dd43c43f9175a965d62ffe2df, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:31,135 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679694871134"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694871134"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694871134"}]},"ts":"1679694871134"} 2023-03-24 21:54:31,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 471fea1dd43c43f9175a965d62ffe2df, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:31,288 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:31,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 471fea1dd43c43f9175a965d62ffe2df, disabling compactions & flushes 2023-03-24 21:54:31,288 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:31,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:31,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. after waiting 0 ms 2023-03-24 21:54:31,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:31,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-24 21:54:31,294 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:31,294 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:31,294 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 471fea1dd43c43f9175a965d62ffe2df to self. 2023-03-24 21:54:31,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:31,296 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=471fea1dd43c43f9175a965d62ffe2df, regionState=CLOSED 2023-03-24 21:54:31,296 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679694871296"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694871296"}]},"ts":"1679694871296"} 2023-03-24 21:54:31,299 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-03-24 21:54:31,299 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 471fea1dd43c43f9175a965d62ffe2df, server=jenkins-hbase9.apache.org,41807,1679694819103 in 161 msec 2023-03-24 21:54:31,299 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=471fea1dd43c43f9175a965d62ffe2df, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=true 2023-03-24 21:54:31,450 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=471fea1dd43c43f9175a965d62ffe2df, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:31,450 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679694871450"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694871450"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694871450"}]},"ts":"1679694871450"} 2023-03-24 21:54:31,452 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 471fea1dd43c43f9175a965d62ffe2df, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:31,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:31,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 471fea1dd43c43f9175a965d62ffe2df, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:54:31,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:31,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:31,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:31,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:31,611 INFO [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:31,612 DEBUG [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1 2023-03-24 21:54:31,612 DEBUG [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1 2023-03-24 21:54:31,613 INFO [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 471fea1dd43c43f9175a965d62ffe2df columnFamilyName family1 2023-03-24 21:54:31,636 DEBUG [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/184a78937414431696866e50d6267d6d_SeqId_53_ 2023-03-24 21:54:31,640 DEBUG [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/family1/4e25cb4240d54d40a9ba752b336d3bf1 2023-03-24 21:54:31,641 INFO [StoreOpener-471fea1dd43c43f9175a965d62ffe2df-1] regionserver.HStore(310): Store=471fea1dd43c43f9175a965d62ffe2df/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:31,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:31,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:31,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:54:31,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 471fea1dd43c43f9175a965d62ffe2df; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59203214, jitterRate=-0.11780336499214172}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:54:31,651 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:54:31,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df., pid=53, masterSystemTime=1679694871603 2023-03-24 21:54:31,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:31,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:54:31,655 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=471fea1dd43c43f9175a965d62ffe2df, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:31,655 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679694871655"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694871655"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694871655"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694871655"}]},"ts":"1679694871655"} 2023-03-24 21:54:31,658 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-03-24 21:54:31,659 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 471fea1dd43c43f9175a965d62ffe2df, server=jenkins-hbase9.apache.org,41807,1679694819103 in 205 msec 2023-03-24 21:54:31,660 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-03-24 21:54:31,660 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=471fea1dd43c43f9175a965d62ffe2df, REOPEN/MOVE in 526 msec 2023-03-24 21:54:31,662 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-03-24 21:54:31,662 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 533 msec 2023-03-24 21:54:31,666 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 553 msec 2023-03-24 21:54:31,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-03-24 21:54:31,668 INFO [Listener at localhost/39907] compactions.TestMobCompactor(278): alter status finished 2023-03-24 21:54:31,669 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-24 21:54:31,671 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-24 21:54:31,672 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-24 21:54:31,672 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:31,672 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695471672 2023-03-24 21:54:31,673 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-24 21:54:31,674 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-24 21:54:31,674 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-24 21:54:31,674 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x5e9c6060 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:54:31,679 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@57a9d3a3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:54:31,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:31,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:31,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:31,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:31,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:31,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:31,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:31,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-24 21:54:31,679 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-24 21:54:31,679 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:31,680 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-24 21:54:31,682 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 10 msec 2023-03-24 21:54:31,737 INFO [Listener at localhost/39907] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=264 (was 270), OpenFileDescriptor=645 (was 631) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=413 (was 388) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=8116 (was 8164) 2023-03-24 21:54:31,750 INFO [Listener at localhost/39907] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=264, OpenFileDescriptor=645, MaxFileDescriptor=60000, SystemLoadAverage=413, ProcessCount=169, AvailableMemoryMB=8115 2023-03-24 21:54:31,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-24 21:54:31,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:31,754 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:54:31,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-03-24 21:54:31,755 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:54:31,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-24 21:54:31,756 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:31,757 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce empty. 2023-03-24 21:54:31,757 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:31,757 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-03-24 21:54:31,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-24 21:54:32,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-24 21:54:32,166 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-24 21:54:32,168 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => 9329e175b349e4e9722958f8463695ce, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:54:32,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-24 21:54:32,583 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:32,583 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing 9329e175b349e4e9722958f8463695ce, disabling compactions & flushes 2023-03-24 21:54:32,583 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:32,583 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:32,583 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. after waiting 0 ms 2023-03-24 21:54:32,583 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:32,583 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:32,583 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:32,586 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:54:32,587 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679694872586"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694872586"}]},"ts":"1679694872586"} 2023-03-24 21:54:32,588 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-24 21:54:32,589 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:54:32,589 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694872589"}]},"ts":"1679694872589"} 2023-03-24 21:54:32,590 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-03-24 21:54:32,593 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=9329e175b349e4e9722958f8463695ce, ASSIGN}] 2023-03-24 21:54:32,595 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=9329e175b349e4e9722958f8463695ce, ASSIGN 2023-03-24 21:54:32,597 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=9329e175b349e4e9722958f8463695ce, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:54:32,749 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=9329e175b349e4e9722958f8463695ce, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:32,749 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679694872749"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694872749"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694872749"}]},"ts":"1679694872749"} 2023-03-24 21:54:32,750 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 9329e175b349e4e9722958f8463695ce, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:32,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-24 21:54:32,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:32,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9329e175b349e4e9722958f8463695ce, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:54:32,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:32,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:32,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:32,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:32,907 INFO [StoreOpener-9329e175b349e4e9722958f8463695ce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:32,908 DEBUG [StoreOpener-9329e175b349e4e9722958f8463695ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1 2023-03-24 21:54:32,908 DEBUG [StoreOpener-9329e175b349e4e9722958f8463695ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1 2023-03-24 21:54:32,908 INFO [StoreOpener-9329e175b349e4e9722958f8463695ce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9329e175b349e4e9722958f8463695ce columnFamilyName family1 2023-03-24 21:54:32,909 INFO [StoreOpener-9329e175b349e4e9722958f8463695ce-1] regionserver.HStore(310): Store=9329e175b349e4e9722958f8463695ce/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:32,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:32,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:32,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:32,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:54:32,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 9329e175b349e4e9722958f8463695ce; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67112698, jitterRate=5.7131052017211914E-5}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:54:32,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:32,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce., pid=57, masterSystemTime=1679694872902 2023-03-24 21:54:32,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:32,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:32,918 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=9329e175b349e4e9722958f8463695ce, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:32,918 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679694872917"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694872917"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694872917"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694872917"}]},"ts":"1679694872917"} 2023-03-24 21:54:32,920 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-03-24 21:54:32,920 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 9329e175b349e4e9722958f8463695ce, server=jenkins-hbase9.apache.org,41807,1679694819103 in 169 msec 2023-03-24 21:54:32,922 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-03-24 21:54:32,922 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=9329e175b349e4e9722958f8463695ce, ASSIGN in 327 msec 2023-03-24 21:54:32,922 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:54:32,922 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694872922"}]},"ts":"1679694872922"} 2023-03-24 21:54:32,923 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-03-24 21:54:32,925 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:54:32,926 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1730 sec 2023-03-24 21:54:33,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-24 21:54:33,861 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-03-24 21:54:33,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:33,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:33,884 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:33,884 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:33,884 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-24 21:54:33,884 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-24 21:54:33,884 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:33,884 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:33,887 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:33,887 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:33,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:33,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:33,887 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:33,887 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:33,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:33,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:33,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:33,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:33,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:33,888 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:33,888 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-24 21:54:33,888 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:33,888 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-24 21:54:33,889 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:33,889 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:33,889 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:33,889 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. started... 2023-03-24 21:54:33,889 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9329e175b349e4e9722958f8463695ce 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-24 21:54:34,309 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511309d5fe83c841c48ed9a71973d77d44b8f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511309d5fe83c841c48ed9a71973d77d44b8f 2023-03-24 21:54:34,713 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/685748aa0108472b83acec3ecffb8596 2023-03-24 21:54:34,718 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/685748aa0108472b83acec3ecffb8596 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/685748aa0108472b83acec3ecffb8596 2023-03-24 21:54:34,722 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/685748aa0108472b83acec3ecffb8596, entries=1000, sequenceid=5, filesize=190.5 K 2023-03-24 21:54:34,723 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 9329e175b349e4e9722958f8463695ce in 834ms, sequenceid=5, compaction requested=false 2023-03-24 21:54:34,723 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:34,724 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:34,724 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:34,724 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:34,724 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:34,724 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,724 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-24 21:54:34,724 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-24 21:54:34,725 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,725 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:34,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:34,726 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,726 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:34,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:34,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:34,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:34,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-24 21:54:34,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5589322d[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:34,727 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-24 21:54:34,727 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,728 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,728 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-24 21:54:34,728 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-24 21:54:34,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,728 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:34,728 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:34,730 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,730 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-24 21:54:34,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:34,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:34,730 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:34,730 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-24 21:54:34,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:34,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:34,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:34,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:34,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-24 21:54:34,732 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-24 21:54:34,732 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:34,732 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:34,732 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,732 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:34,734 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,735 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,735 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:34,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:34,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:34,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:34,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:34,735 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:34,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:34,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,736 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:34,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,738 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,738 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,738 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:34,738 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:34,738 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:34,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:34,738 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:34,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:34,738 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,738 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:34,739 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,739 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,739 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,739 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:34,739 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:34,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:34,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:34,839 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:34,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:34,844 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:34,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:34,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,846 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:34,846 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:34,846 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-24 21:54:34,846 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-24 21:54:34,846 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,846 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,851 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,851 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:34,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:34,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:34,852 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:34,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,852 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:34,852 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:34,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,852 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,853 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-24 21:54:34,853 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:34,853 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-24 21:54:34,853 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:34,853 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-24 21:54:34,853 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:34,853 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:34,853 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:34,853 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. started... 2023-03-24 21:54:34,853 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9329e175b349e4e9722958f8463695ce 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-24 21:54:35,266 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511280e59b363b62242b1b78c5bd8a401c17d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511280e59b363b62242b1b78c5bd8a401c17d 2023-03-24 21:54:35,671 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/9bf3ae5ba8e040c5827f9ea4e2e46385 2023-03-24 21:54:35,675 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/9bf3ae5ba8e040c5827f9ea4e2e46385 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/9bf3ae5ba8e040c5827f9ea4e2e46385 2023-03-24 21:54:35,678 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-24 21:54:35,680 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/9bf3ae5ba8e040c5827f9ea4e2e46385, entries=1, sequenceid=9, filesize=5.0 K 2023-03-24 21:54:35,680 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 9329e175b349e4e9722958f8463695ce in 827ms, sequenceid=9, compaction requested=false 2023-03-24 21:54:35,681 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:35,681 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:35,681 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:35,681 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:35,681 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,681 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-24 21:54:35,681 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-24 21:54:35,683 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,683 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:35,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:35,683 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,683 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:35,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:35,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:35,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:35,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-24 21:54:35,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3774d544[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:35,685 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-24 21:54:35,685 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,686 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,686 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-24 21:54:35,686 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-24 21:54:35,686 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:35,686 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,686 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:35,687 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,687 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-24 21:54:35,687 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:35,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:35,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:35,687 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-24 21:54:35,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:35,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:35,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:35,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:35,689 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-24 21:54:35,689 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-24 21:54:35,689 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:35,689 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:35,690 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,690 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:35,692 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,692 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,692 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:35,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:35,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:35,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:35,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:35,692 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:35,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:35,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,695 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:35,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,697 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,697 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:35,697 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,697 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:35,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:35,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:35,697 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:35,697 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:35,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:35,697 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,697 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:35,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:35,697 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,698 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,698 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:35,698 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:35,798 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:35,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:35,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:35,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:35,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,804 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:35,804 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:35,804 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-24 21:54:35,804 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-24 21:54:35,805 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,805 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,806 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:35,806 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:35,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:35,806 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:35,806 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:35,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,807 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,807 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,807 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-24 21:54:35,807 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:35,807 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-24 21:54:35,807 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:35,807 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-24 21:54:35,807 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:35,808 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:35,808 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:35,808 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. started... 2023-03-24 21:54:35,808 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9329e175b349e4e9722958f8463695ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:36,220 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201cb4b45bd85174b50ab2b365748593c24 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201cb4b45bd85174b50ab2b365748593c24 2023-03-24 21:54:36,625 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/0918ca49ed92429288840e33c84a73bb 2023-03-24 21:54:36,630 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/0918ca49ed92429288840e33c84a73bb as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/0918ca49ed92429288840e33c84a73bb 2023-03-24 21:54:36,633 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/0918ca49ed92429288840e33c84a73bb, entries=1, sequenceid=13, filesize=5.0 K 2023-03-24 21:54:36,634 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9329e175b349e4e9722958f8463695ce in 826ms, sequenceid=13, compaction requested=true 2023-03-24 21:54:36,634 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:36,634 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:36,634 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:36,634 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:36,634 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,634 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-24 21:54:36,634 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-24 21:54:36,636 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,636 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:36,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:36,636 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,636 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:36,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:36,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:36,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:36,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-24 21:54:36,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75ac9254[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:36,637 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-24 21:54:36,637 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,639 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,640 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-24 21:54:36,640 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-24 21:54:36,640 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:36,640 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,640 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:36,641 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,641 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-24 21:54:36,641 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:36,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:36,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:36,641 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-24 21:54:36,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:36,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:36,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:36,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:36,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-24 21:54:36,643 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-24 21:54:36,643 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:36,643 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:36,643 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,643 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:36,644 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,644 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:36,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:36,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,644 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:36,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:36,645 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:36,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,646 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:36,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:36,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,648 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:36,648 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:36,648 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:36,648 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-24 21:54:36,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:36,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:36,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:36,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,648 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:36,648 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:36,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:36,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:36,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,748 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:36,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:36,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:36,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:36,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,754 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:36,754 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:36,754 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-24 21:54:36,754 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-24 21:54:36,755 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,755 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,757 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,757 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:36,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:36,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:36,758 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:36,758 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:36,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:36,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,758 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-24 21:54:36,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:36,759 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-24 21:54:36,759 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:36,759 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-24 21:54:36,759 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:36,759 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:36,759 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:36,759 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. started... 2023-03-24 21:54:36,759 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9329e175b349e4e9722958f8463695ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:37,172 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205a572fcd6b4e94a6dbb876da1077dd47a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205a572fcd6b4e94a6dbb876da1077dd47a 2023-03-24 21:54:37,577 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/76dc5fc7e7864444a8d9a5ee67ff8a1b 2023-03-24 21:54:37,582 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/76dc5fc7e7864444a8d9a5ee67ff8a1b as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/76dc5fc7e7864444a8d9a5ee67ff8a1b 2023-03-24 21:54:37,586 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/76dc5fc7e7864444a8d9a5ee67ff8a1b, entries=1, sequenceid=17, filesize=5.0 K 2023-03-24 21:54:37,586 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9329e175b349e4e9722958f8463695ce in 827ms, sequenceid=17, compaction requested=true 2023-03-24 21:54:37,586 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:37,586 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:37,586 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:37,587 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:37,587 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,587 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-24 21:54:37,587 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-24 21:54:37,588 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,588 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:37,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:37,588 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,588 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:37,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:37,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:37,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:37,589 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-24 21:54:37,589 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60c411e8[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:37,589 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-24 21:54:37,590 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,590 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,591 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-24 21:54:37,591 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-24 21:54:37,591 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:37,591 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,591 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:37,598 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-24 21:54:37,598 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,598 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:37,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:37,598 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-24 21:54:37,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:37,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:37,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:37,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:37,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:37,600 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-24 21:54:37,600 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-24 21:54:37,600 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:37,600 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:37,600 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,600 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:37,602 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,602 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:37,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:37,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,602 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:37,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:37,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:37,602 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:37,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:37,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,603 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:37,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,605 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,605 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,605 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:37,605 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:37,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:37,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:37,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:37,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-24 21:54:37,605 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:37,605 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:37,605 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,605 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,605 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:37,606 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:37,605 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,606 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:37,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:37,706 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:37,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:37,710 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:37,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:37,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,712 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:37,712 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:37,712 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-24 21:54:37,712 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-24 21:54:37,712 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,712 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,713 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,713 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:37,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:37,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:37,713 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:37,713 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:37,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,714 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:37,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,714 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,714 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-24 21:54:37,714 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:37,714 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-24 21:54:37,714 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:37,715 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-24 21:54:37,715 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:37,715 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:37,715 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:37,715 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. started... 2023-03-24 21:54:37,715 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9329e175b349e4e9722958f8463695ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:38,126 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228ca3ed39727af4f85bac38058883bbf7b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228ca3ed39727af4f85bac38058883bbf7b 2023-03-24 21:54:38,531 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/295301d69793458f84bc31ffedca2a71 2023-03-24 21:54:38,536 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/295301d69793458f84bc31ffedca2a71 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/295301d69793458f84bc31ffedca2a71 2023-03-24 21:54:38,539 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/295301d69793458f84bc31ffedca2a71, entries=1, sequenceid=21, filesize=5.0 K 2023-03-24 21:54:38,540 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9329e175b349e4e9722958f8463695ce in 825ms, sequenceid=21, compaction requested=true 2023-03-24 21:54:38,540 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:38,540 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:38,540 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:38,540 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:38,540 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,540 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-24 21:54:38,540 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-24 21:54:38,543 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,543 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:38,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:38,543 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,543 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:38,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:38,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:38,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:38,544 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-24 21:54:38,544 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cc05f87[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:38,544 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-24 21:54:38,544 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,545 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,545 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-24 21:54:38,545 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-24 21:54:38,546 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:38,546 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,546 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:38,547 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-24 21:54:38,547 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:38,547 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,547 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-24 21:54:38,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:38,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:38,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:38,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:38,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:38,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:38,549 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-24 21:54:38,549 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-24 21:54:38,549 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:38,549 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:38,549 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,549 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:38,550 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,550 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:38,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:38,550 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:38,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:38,551 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:38,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:38,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:38,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,552 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:38,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,555 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,555 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:38,555 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,555 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:38,555 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:38,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:38,555 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:38,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:38,555 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:38,555 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,556 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,556 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:38,556 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:38,556 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:38,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:38,656 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:38,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:38,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:38,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:38,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,662 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:38,662 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:38,662 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-24 21:54:38,662 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-24 21:54:38,662 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,662 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,664 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:38,664 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:38,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:38,664 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:38,664 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:38,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,664 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:38,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,665 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,665 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-24 21:54:38,665 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:38,665 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-24 21:54:38,665 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:38,665 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-24 21:54:38,665 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:38,665 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:38,665 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:38,665 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. started... 2023-03-24 21:54:38,665 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9329e175b349e4e9722958f8463695ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:39,077 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512311c5e816980c0447b8635621b66fb9542 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512311c5e816980c0447b8635621b66fb9542 2023-03-24 21:54:39,083 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/2528c0bcc35c4db58b6b510bafe96b4d 2023-03-24 21:54:39,087 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/2528c0bcc35c4db58b6b510bafe96b4d as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/2528c0bcc35c4db58b6b510bafe96b4d 2023-03-24 21:54:39,091 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/2528c0bcc35c4db58b6b510bafe96b4d, entries=1, sequenceid=25, filesize=5.0 K 2023-03-24 21:54:39,092 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9329e175b349e4e9722958f8463695ce in 427ms, sequenceid=25, compaction requested=true 2023-03-24 21:54:39,092 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:39,092 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:39,092 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:39,092 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:39,092 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,092 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-24 21:54:39,092 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-24 21:54:39,093 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,093 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:39,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:39,094 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,094 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:39,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:39,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:39,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:39,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-24 21:54:39,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d0f7bae[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:39,095 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-24 21:54:39,095 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,096 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,096 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-24 21:54:39,096 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-24 21:54:39,096 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,096 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:39,096 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:39,098 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-24 21:54:39,098 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,098 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:39,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,098 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-24 21:54:39,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:39,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:39,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:39,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:39,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:39,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:39,100 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-24 21:54:39,100 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-24 21:54:39,101 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:39,101 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:39,101 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,101 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:39,102 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,102 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:39,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,102 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:39,103 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:39,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:39,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:39,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,104 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:39,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,108 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,108 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:39,108 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:39,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:39,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:39,108 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:39,108 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,108 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:39,108 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:39,108 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,108 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,108 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:39,109 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:39,108 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:39,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:39,209 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:39,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:39,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:39,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:39,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,215 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:39,215 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:39,215 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-24 21:54:39,215 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-24 21:54:39,215 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,215 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,220 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:39,220 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:39,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:39,220 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:39,220 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:39,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:39,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,221 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,221 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-24 21:54:39,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:39,221 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-24 21:54:39,221 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:39,222 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-24 21:54:39,222 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:39,222 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:39,222 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:39,222 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. started... 2023-03-24 21:54:39,222 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9329e175b349e4e9722958f8463695ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:39,634 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010180e58f1c3ba4442e83a2783b828c01a3 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010180e58f1c3ba4442e83a2783b828c01a3 2023-03-24 21:54:40,039 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/1a5fdf0b9c5e426289db69e936be741f 2023-03-24 21:54:40,044 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/1a5fdf0b9c5e426289db69e936be741f as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/1a5fdf0b9c5e426289db69e936be741f 2023-03-24 21:54:40,048 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/1a5fdf0b9c5e426289db69e936be741f, entries=1, sequenceid=29, filesize=5.0 K 2023-03-24 21:54:40,049 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9329e175b349e4e9722958f8463695ce in 827ms, sequenceid=29, compaction requested=true 2023-03-24 21:54:40,049 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:40,049 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:40,049 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:40,049 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:40,049 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,049 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-24 21:54:40,049 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-24 21:54:40,051 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,051 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:40,051 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,051 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:40,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:40,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:40,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-24 21:54:40,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ca07a69[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:40,053 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-24 21:54:40,053 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,054 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,054 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,054 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:40,054 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-24 21:54:40,054 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-24 21:54:40,054 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:40,057 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-24 21:54:40,057 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:40,057 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,057 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-24 21:54:40,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:40,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:40,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:40,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:40,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:40,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:40,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-24 21:54:40,059 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-24 21:54:40,059 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:40,059 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:40,059 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,059 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:40,061 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,061 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:40,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:40,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,061 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:40,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:40,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:40,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:40,062 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:40,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,063 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:40,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,064 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,064 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:40,064 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:40,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:40,065 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:40,065 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:40,065 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-24 21:54:40,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:40,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:40,065 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:40,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:40,065 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,065 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,065 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,065 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,065 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:40,065 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:40,165 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:40,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:40,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:40,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:40,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,172 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:40,172 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:40,172 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-24 21:54:40,172 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-24 21:54:40,172 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,172 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,174 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,174 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:40,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:40,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:40,174 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:40,174 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:40,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,175 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:40,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,175 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-24 21:54:40,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:40,175 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-24 21:54:40,175 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:40,176 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-24 21:54:40,176 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:40,176 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:40,176 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:40,176 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. started... 2023-03-24 21:54:40,176 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9329e175b349e4e9722958f8463695ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:40,589 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d95a30dae5c6476a97f0fa5138524ae4 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103d95a30dae5c6476a97f0fa5138524ae4 2023-03-24 21:54:40,994 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/dfa9433d6a7442bd83a8a79da3937d25 2023-03-24 21:54:40,998 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/dfa9433d6a7442bd83a8a79da3937d25 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/dfa9433d6a7442bd83a8a79da3937d25 2023-03-24 21:54:41,002 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/dfa9433d6a7442bd83a8a79da3937d25, entries=1, sequenceid=33, filesize=5.0 K 2023-03-24 21:54:41,002 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9329e175b349e4e9722958f8463695ce in 826ms, sequenceid=33, compaction requested=true 2023-03-24 21:54:41,002 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:41,002 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:41,003 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:41,003 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:41,003 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,003 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-24 21:54:41,003 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-24 21:54:41,007 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,007 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:41,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:41,007 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,007 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:41,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:41,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:41,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:41,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-24 21:54:41,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3cab7f44[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:41,008 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-24 21:54:41,008 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,009 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,009 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-24 21:54:41,010 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,010 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:41,010 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-24 21:54:41,010 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:41,011 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,011 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-24 21:54:41,011 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:41,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:41,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:41,011 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-24 21:54:41,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:41,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:41,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:41,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:41,013 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-24 21:54:41,013 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-24 21:54:41,013 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:41,013 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:41,013 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,013 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:41,015 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,015 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,015 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:41,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:41,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:41,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:41,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:41,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:41,016 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:41,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,017 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:41,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,018 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,018 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:41,018 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:41,018 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:41,019 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:41,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:41,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:41,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:41,019 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,019 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,019 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,019 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,019 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,019 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:41,019 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:41,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:41,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:41,119 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:41,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:41,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:41,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:41,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,125 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:41,126 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:41,126 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-24 21:54:41,126 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-24 21:54:41,126 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,126 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,127 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,127 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:41,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:41,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:41,128 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,128 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:41,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,128 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:41,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,128 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,129 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-24 21:54:41,129 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,129 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-24 21:54:41,129 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:41,129 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-24 21:54:41,129 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:41,129 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:41,129 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:41,129 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. started... 2023-03-24 21:54:41,129 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9329e175b349e4e9722958f8463695ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:41,541 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907ce40beb3c20e4d0bb5aa22afb9f9184c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907ce40beb3c20e4d0bb5aa22afb9f9184c 2023-03-24 21:54:41,946 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/027eacd1cbdf44f882cd851310564e9e 2023-03-24 21:54:41,951 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/027eacd1cbdf44f882cd851310564e9e as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/027eacd1cbdf44f882cd851310564e9e 2023-03-24 21:54:41,955 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/027eacd1cbdf44f882cd851310564e9e, entries=1, sequenceid=37, filesize=5.0 K 2023-03-24 21:54:41,956 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9329e175b349e4e9722958f8463695ce in 827ms, sequenceid=37, compaction requested=true 2023-03-24 21:54:41,956 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:41,956 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:41,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:41,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:41,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-24 21:54:41,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-24 21:54:41,959 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,959 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:41,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:41,959 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,959 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:41,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:41,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:41,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:41,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-24 21:54:41,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42e93331[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:41,960 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-24 21:54:41,960 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,961 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,961 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-24 21:54:41,961 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-24 21:54:41,961 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:41,962 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,962 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:41,963 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,963 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-24 21:54:41,963 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:41,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:41,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:41,963 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-24 21:54:41,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:41,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:41,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:41,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:41,965 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-24 21:54:41,965 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-24 21:54:41,965 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:41,965 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:41,965 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,965 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:41,967 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,967 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,967 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:41,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:41,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:41,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:41,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:41,967 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:41,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:41,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,968 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:41,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,971 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,971 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,971 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,971 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:41,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:41,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:41,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:41,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:41,971 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:41,971 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:41,971 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:41,971 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:41,971 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-24 21:54:41,972 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:41,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:41,971 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:41,972 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:41,972 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,072 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:42,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:42,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:42,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:42,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,078 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:42,078 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:42,078 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-24 21:54:42,078 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-24 21:54:42,078 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,078 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,080 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:42,080 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:42,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:42,080 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,081 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:42,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:42,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-24 21:54:42,082 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,082 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-24 21:54:42,082 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:42,082 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-24 21:54:42,082 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:54:42,082 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:42,082 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:42,082 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. started... 2023-03-24 21:54:42,082 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9329e175b349e4e9722958f8463695ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:54:42,498 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112013676f6ba4e6440fbaf93637088e79f7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112013676f6ba4e6440fbaf93637088e79f7 2023-03-24 21:54:42,503 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/f3a4094c069745cd9a13724639692cb4 2023-03-24 21:54:42,507 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/f3a4094c069745cd9a13724639692cb4 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/f3a4094c069745cd9a13724639692cb4 2023-03-24 21:54:42,510 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/f3a4094c069745cd9a13724639692cb4, entries=1, sequenceid=41, filesize=5.0 K 2023-03-24 21:54:42,511 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9329e175b349e4e9722958f8463695ce in 429ms, sequenceid=41, compaction requested=true 2023-03-24 21:54:42,511 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:42,512 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:42,512 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:54:42,512 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:54:42,512 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,512 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-24 21:54:42,512 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-24 21:54:42,513 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,513 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:42,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:42,513 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,514 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:42,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:42,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:42,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:42,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-24 21:54:42,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@542df2c9[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:42,515 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-24 21:54:42,515 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,516 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,516 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-24 21:54:42,516 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-24 21:54:42,516 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,516 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:42,516 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:42,517 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-24 21:54:42,517 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,517 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:42,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:42,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:42,517 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-24 21:54:42,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:42,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:42,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:42,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:42,519 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-24 21:54:42,519 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-24 21:54:42,519 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:42,519 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:42,519 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,520 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:42,522 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,522 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:42,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:42,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,522 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:42,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:42,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:42,522 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:42,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:42,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,523 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:42,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,525 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,525 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,525 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,525 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:42,525 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:42,525 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:42,525 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:42,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:42,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:42,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-24 21:54:42,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:42,525 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,526 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-24 21:54:42,526 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:42,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:42,526 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,526 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:42,526 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:42,626 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:42,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:42,628 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:42,629 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-24 21:54:42,630 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-24 21:54:42,631 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:42,631 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695482631 2023-03-24 21:54:42,632 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-24 21:54:42,633 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-24 21:54:42,633 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-24 21:54:42,634 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x692257b5 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:54:42,638 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5521039, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:54:42,638 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:43,474 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d5b0e21218604287b1604a0ce7addf66 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103d5b0e21218604287b1604a0ce7addf66 2023-03-24 21:54:43,475 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-24 21:54:43,488 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:54:43,489 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:59454, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:54:43,498 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:54:43,499 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:55034, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:54:43,510 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/fbc7a670acf74d188f245dbed50ff112 first=Optional[r3] last=Optional[r6] 2023-03-24 21:54:43,511 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/fbc7a670acf74d188f245dbed50ff112}] 2023-03-24 21:54:43,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/fbc7a670acf74d188f245dbed50ff112 for inclusion in 9329e175b349e4e9722958f8463695ce/family1 2023-03-24 21:54:43,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-24 21:54:43,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:54:43,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:43,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/fbc7a670acf74d188f245dbed50ff112 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__o231s4t17gsi1889kgk9s5aoa71efjq2rvr9d9fsnguav3eqr8u8r3a0ta9cj4pa/family1/fbc7a670acf74d188f245dbed50ff112 2023-03-24 21:54:43,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__o231s4t17gsi1889kgk9s5aoa71efjq2rvr9d9fsnguav3eqr8u8r3a0ta9cj4pa/family1/fbc7a670acf74d188f245dbed50ff112 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/5c5a205e7a9043fb8e20ef28576c9133_SeqId_44_ 2023-03-24 21:54:43,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__o231s4t17gsi1889kgk9s5aoa71efjq2rvr9d9fsnguav3eqr8u8r3a0ta9cj4pa/family1/fbc7a670acf74d188f245dbed50ff112 into 9329e175b349e4e9722958f8463695ce/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/5c5a205e7a9043fb8e20ef28576c9133_SeqId_44_ - updating store file list. 2023-03-24 21:54:43,528 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/5c5a205e7a9043fb8e20ef28576c9133_SeqId_44_ into 9329e175b349e4e9722958f8463695ce/family1 2023-03-24 21:54:43,528 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__o231s4t17gsi1889kgk9s5aoa71efjq2rvr9d9fsnguav3eqr8u8r3a0ta9cj4pa/family1/fbc7a670acf74d188f245dbed50ff112 into 9329e175b349e4e9722958f8463695ce/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/5c5a205e7a9043fb8e20ef28576c9133_SeqId_44_) 2023-03-24 21:54:43,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__o231s4t17gsi1889kgk9s5aoa71efjq2rvr9d9fsnguav3eqr8u8r3a0ta9cj4pa/family1/fbc7a670acf74d188f245dbed50ff112 2023-03-24 21:54:43,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-24 21:54:43,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-24 21:54:43,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(1912): 9329e175b349e4e9722958f8463695ce/family1 is initiating minor compaction 2023-03-24 21:54:43,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3a0e6f1b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:43,533 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 9329e175b349e4e9722958f8463695ce/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:43,534 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/9bf3ae5ba8e040c5827f9ea4e2e46385, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/0918ca49ed92429288840e33c84a73bb, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/76dc5fc7e7864444a8d9a5ee67ff8a1b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/295301d69793458f84bc31ffedca2a71, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/2528c0bcc35c4db58b6b510bafe96b4d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/1a5fdf0b9c5e426289db69e936be741f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/dfa9433d6a7442bd83a8a79da3937d25, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/027eacd1cbdf44f882cd851310564e9e, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/f3a4094c069745cd9a13724639692cb4, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/5c5a205e7a9043fb8e20ef28576c9133_SeqId_44_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp, totalSize=50.3 K 2023-03-24 21:54:43,534 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 9bf3ae5ba8e040c5827f9ea4e2e46385, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-24 21:54:43,534 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 0918ca49ed92429288840e33c84a73bb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-24 21:54:43,534 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 76dc5fc7e7864444a8d9a5ee67ff8a1b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-24 21:54:43,534 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 295301d69793458f84bc31ffedca2a71, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-24 21:54:43,534 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 2528c0bcc35c4db58b6b510bafe96b4d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-24 21:54:43,534 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 1a5fdf0b9c5e426289db69e936be741f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-24 21:54:43,534 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting dfa9433d6a7442bd83a8a79da3937d25, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-24 21:54:43,534 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 027eacd1cbdf44f882cd851310564e9e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-24 21:54:43,534 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting f3a4094c069745cd9a13724639692cb4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-24 21:54:43,534 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 5c5a205e7a9043fb8e20ef28576c9133_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-24 21:54:43,536 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-24 21:54:43,537 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:43,539 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228ca3ed39727af4f85bac38058883bbf7b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228ca3ed39727af4f85bac38058883bbf7b 2023-03-24 21:54:43,541 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512311c5e816980c0447b8635621b66fb9542 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512311c5e816980c0447b8635621b66fb9542 2023-03-24 21:54:43,543 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010180e58f1c3ba4442e83a2783b828c01a3 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010180e58f1c3ba4442e83a2783b828c01a3 2023-03-24 21:54:43,544 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103d95a30dae5c6476a97f0fa5138524ae4 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103d95a30dae5c6476a97f0fa5138524ae4 2023-03-24 21:54:43,544 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-03-24 21:54:43,544 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-24 21:54:43,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:43,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:43,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:43,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:43,546 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:43,546 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:43,546 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:43,546 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:43,584 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9329e175b349e4e9722958f8463695ce#family1#compaction#72 average throughput is 0.16 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-03-24 21:54:44,014 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205ef0a66f1d92d44ef96f24fc29541dc3f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205ef0a66f1d92d44ef96f24fc29541dc3f 2023-03-24 21:54:44,014 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/a77dd57232fb4d6a8bc087ae5fa89687 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/a77dd57232fb4d6a8bc087ae5fa89687 2023-03-24 21:54:44,015 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-24 21:54:44,019 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 9329e175b349e4e9722958f8463695ce/family1 of 9329e175b349e4e9722958f8463695ce into a77dd57232fb4d6a8bc087ae5fa89687(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:54:44,019 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:44,020 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1679694883533; duration=0sec 2023-03-24 21:54:44,020 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:44,039 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ba8147b2e3c84175aaac3b64c000b9ff first=Optional[r00] last=Optional[r2] 2023-03-24 21:54:44,039 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ba8147b2e3c84175aaac3b64c000b9ff}] 2023-03-24 21:54:44,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ba8147b2e3c84175aaac3b64c000b9ff for inclusion in 9329e175b349e4e9722958f8463695ce/family1 2023-03-24 21:54:44,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-24 21:54:44,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:54:44,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:44,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ba8147b2e3c84175aaac3b64c000b9ff to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__lrn97kcgaj6acao57uoa41km8ve3ppsso53d454osv34emc6av1lj0non9g48i71/family1/ba8147b2e3c84175aaac3b64c000b9ff 2023-03-24 21:54:44,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__lrn97kcgaj6acao57uoa41km8ve3ppsso53d454osv34emc6av1lj0non9g48i71/family1/ba8147b2e3c84175aaac3b64c000b9ff as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/16b111ee43d442bb93bfef8c3840dd3f_SeqId_47_ 2023-03-24 21:54:44,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__lrn97kcgaj6acao57uoa41km8ve3ppsso53d454osv34emc6av1lj0non9g48i71/family1/ba8147b2e3c84175aaac3b64c000b9ff into 9329e175b349e4e9722958f8463695ce/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/16b111ee43d442bb93bfef8c3840dd3f_SeqId_47_ - updating store file list. 2023-03-24 21:54:44,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/16b111ee43d442bb93bfef8c3840dd3f_SeqId_47_ into 9329e175b349e4e9722958f8463695ce/family1 2023-03-24 21:54:44,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__lrn97kcgaj6acao57uoa41km8ve3ppsso53d454osv34emc6av1lj0non9g48i71/family1/ba8147b2e3c84175aaac3b64c000b9ff into 9329e175b349e4e9722958f8463695ce/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/16b111ee43d442bb93bfef8c3840dd3f_SeqId_47_) 2023-03-24 21:54:44,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__lrn97kcgaj6acao57uoa41km8ve3ppsso53d454osv34emc6av1lj0non9g48i71/family1/ba8147b2e3c84175aaac3b64c000b9ff 2023-03-24 21:54:44,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-24 21:54:44,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-24 21:54:44,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(1912): 9329e175b349e4e9722958f8463695ce/family1 is initiating minor compaction (all files) 2023-03-24 21:54:44,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@545c1e7e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:44,052 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 9329e175b349e4e9722958f8463695ce/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:44,053 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/685748aa0108472b83acec3ecffb8596, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/a77dd57232fb4d6a8bc087ae5fa89687, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/16b111ee43d442bb93bfef8c3840dd3f_SeqId_47_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp, totalSize=388.1 K 2023-03-24 21:54:44,053 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 685748aa0108472b83acec3ecffb8596, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841999 2023-03-24 21:54:44,053 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting a77dd57232fb4d6a8bc087ae5fa89687, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241999 2023-03-24 21:54:44,054 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 16b111ee43d442bb93bfef8c3840dd3f_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878841999 2023-03-24 21:54:44,054 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-24 21:54:44,055 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:44,056 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511309d5fe83c841c48ed9a71973d77d44b8f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511309d5fe83c841c48ed9a71973d77d44b8f 2023-03-24 21:54:44,057 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201cb4b45bd85174b50ab2b365748593c24 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201cb4b45bd85174b50ab2b365748593c24 2023-03-24 21:54:44,058 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205a572fcd6b4e94a6dbb876da1077dd47a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205a572fcd6b4e94a6dbb876da1077dd47a 2023-03-24 21:54:44,058 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-03-24 21:54:44,058 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-24 21:54:44,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:44,059 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-24 21:54:44,059 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:44,060 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-24 21:54:44,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.4310 sec 2023-03-24 21:54:44,066 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9329e175b349e4e9722958f8463695ce#family1#compaction#73 average throughput is 29.94 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-03-24 21:54:44,070 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/9bf3ae5ba8e040c5827f9ea4e2e46385, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/0918ca49ed92429288840e33c84a73bb, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/76dc5fc7e7864444a8d9a5ee67ff8a1b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/295301d69793458f84bc31ffedca2a71, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/2528c0bcc35c4db58b6b510bafe96b4d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/1a5fdf0b9c5e426289db69e936be741f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/dfa9433d6a7442bd83a8a79da3937d25, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/027eacd1cbdf44f882cd851310564e9e, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/f3a4094c069745cd9a13724639692cb4, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/5c5a205e7a9043fb8e20ef28576c9133_SeqId_44_] to archive 2023-03-24 21:54:44,070 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:44,072 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/9bf3ae5ba8e040c5827f9ea4e2e46385 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/9bf3ae5ba8e040c5827f9ea4e2e46385 2023-03-24 21:54:44,073 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/0918ca49ed92429288840e33c84a73bb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/0918ca49ed92429288840e33c84a73bb 2023-03-24 21:54:44,074 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/76dc5fc7e7864444a8d9a5ee67ff8a1b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/76dc5fc7e7864444a8d9a5ee67ff8a1b 2023-03-24 21:54:44,075 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/295301d69793458f84bc31ffedca2a71 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/295301d69793458f84bc31ffedca2a71 2023-03-24 21:54:44,076 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/2528c0bcc35c4db58b6b510bafe96b4d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/2528c0bcc35c4db58b6b510bafe96b4d 2023-03-24 21:54:44,077 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/1a5fdf0b9c5e426289db69e936be741f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/1a5fdf0b9c5e426289db69e936be741f 2023-03-24 21:54:44,078 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/dfa9433d6a7442bd83a8a79da3937d25 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/dfa9433d6a7442bd83a8a79da3937d25 2023-03-24 21:54:44,079 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/027eacd1cbdf44f882cd851310564e9e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/027eacd1cbdf44f882cd851310564e9e 2023-03-24 21:54:44,080 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/f3a4094c069745cd9a13724639692cb4 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/f3a4094c069745cd9a13724639692cb4 2023-03-24 21:54:44,081 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/5c5a205e7a9043fb8e20ef28576c9133_SeqId_44_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/5c5a205e7a9043fb8e20ef28576c9133_SeqId_44_ 2023-03-24 21:54:44,087 DEBUG [hfile_cleaner-dir-scan-pool-1] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1 is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1 is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1 is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 17 more 2023-03-24 21:54:44,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$12(2454): Client=jenkins//172.31.2.10 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-24 21:54:44,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:44,539 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-24 21:54:44,540 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-24 21:54:44,540 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-24 21:54:44,542 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-24 21:54:44,544 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=9329e175b349e4e9722958f8463695ce, REOPEN/MOVE}] 2023-03-24 21:54:44,544 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=9329e175b349e4e9722958f8463695ce, REOPEN/MOVE 2023-03-24 21:54:44,545 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=9329e175b349e4e9722958f8463695ce, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:44,545 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679694884545"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694884545"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694884545"}]},"ts":"1679694884545"} 2023-03-24 21:54:44,546 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 9329e175b349e4e9722958f8463695ce, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:44,698 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:44,698 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 9329e175b349e4e9722958f8463695ce, disabling compactions & flushes 2023-03-24 21:54:44,698 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:45,286 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/.tmp/family1/821d4b2e994e451f86c9364a899be68d as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/821d4b2e994e451f86c9364a899be68d 2023-03-24 21:54:45,291 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 9329e175b349e4e9722958f8463695ce/family1 of 9329e175b349e4e9722958f8463695ce into 821d4b2e994e451f86c9364a899be68d(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-24 21:54:45,291 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:45,291 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:45,292 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1679694884052; duration=1sec 2023-03-24 21:54:45,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:45,292 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:45,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. after waiting 0 ms 2023-03-24 21:54:45,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:45,292 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/685748aa0108472b83acec3ecffb8596, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/a77dd57232fb4d6a8bc087ae5fa89687, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/16b111ee43d442bb93bfef8c3840dd3f_SeqId_47_] to archive 2023-03-24 21:54:45,293 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:45,295 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/685748aa0108472b83acec3ecffb8596 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/685748aa0108472b83acec3ecffb8596 2023-03-24 21:54:45,296 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/a77dd57232fb4d6a8bc087ae5fa89687 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/a77dd57232fb4d6a8bc087ae5fa89687 2023-03-24 21:54:45,298 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/16b111ee43d442bb93bfef8c3840dd3f_SeqId_47_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/16b111ee43d442bb93bfef8c3840dd3f_SeqId_47_ 2023-03-24 21:54:45,302 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-24 21:54:45,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:45,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:45,303 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 9329e175b349e4e9722958f8463695ce to self. 2023-03-24 21:54:45,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:45,305 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=9329e175b349e4e9722958f8463695ce, regionState=CLOSED 2023-03-24 21:54:45,305 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679694885305"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694885305"}]},"ts":"1679694885305"} 2023-03-24 21:54:45,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-03-24 21:54:45,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 9329e175b349e4e9722958f8463695ce, server=jenkins-hbase9.apache.org,41807,1679694819103 in 760 msec 2023-03-24 21:54:45,308 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=9329e175b349e4e9722958f8463695ce, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=true 2023-03-24 21:54:45,459 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=9329e175b349e4e9722958f8463695ce, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:45,459 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679694885459"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694885459"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694885459"}]},"ts":"1679694885459"} 2023-03-24 21:54:45,461 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 9329e175b349e4e9722958f8463695ce, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:45,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:45,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9329e175b349e4e9722958f8463695ce, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:54:45,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:45,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:45,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:45,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:45,618 INFO [StoreOpener-9329e175b349e4e9722958f8463695ce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:45,619 DEBUG [StoreOpener-9329e175b349e4e9722958f8463695ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1 2023-03-24 21:54:45,619 DEBUG [StoreOpener-9329e175b349e4e9722958f8463695ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1 2023-03-24 21:54:45,619 INFO [StoreOpener-9329e175b349e4e9722958f8463695ce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9329e175b349e4e9722958f8463695ce columnFamilyName family1 2023-03-24 21:54:45,625 DEBUG [StoreOpener-9329e175b349e4e9722958f8463695ce-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/821d4b2e994e451f86c9364a899be68d 2023-03-24 21:54:45,625 INFO [StoreOpener-9329e175b349e4e9722958f8463695ce-1] regionserver.HStore(310): Store=9329e175b349e4e9722958f8463695ce/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:45,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:45,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:45,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:45,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 9329e175b349e4e9722958f8463695ce; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69503264, jitterRate=0.03567934036254883}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:54:45,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:45,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce., pid=63, masterSystemTime=1679694885612 2023-03-24 21:54:45,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:45,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:45,634 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=9329e175b349e4e9722958f8463695ce, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:45,634 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679694885634"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694885634"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694885634"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694885634"}]},"ts":"1679694885634"} 2023-03-24 21:54:45,637 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-03-24 21:54:45,637 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 9329e175b349e4e9722958f8463695ce, server=jenkins-hbase9.apache.org,41807,1679694819103 in 175 msec 2023-03-24 21:54:45,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-03-24 21:54:45,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=9329e175b349e4e9722958f8463695ce, REOPEN/MOVE in 1.0930 sec 2023-03-24 21:54:45,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-03-24 21:54:45,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0980 sec 2023-03-24 21:54:45,642 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.5120 sec 2023-03-24 21:54:45,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-03-24 21:54:45,645 INFO [Listener at localhost/39907] compactions.TestMobCompactor(278): alter status finished 2023-03-24 21:54:45,645 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:45,647 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-24 21:54:45,647 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-24 21:54:45,648 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695485648 2023-03-24 21:54:45,648 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:45,649 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-24 21:54:45,649 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-24 21:54:45,649 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-24 21:54:45,650 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x548b2894 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:54:45,664 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3adaeb09, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:54:45,664 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:45,664 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:45,664 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:45,665 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:45,665 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:46,106 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112897aab3df4967482d97841958e18ee634 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112897aab3df4967482d97841958e18ee634 2023-03-24 21:54:46,107 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-24 21:54:46,137 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:54:46,138 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:59460, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:54:46,150 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:54:46,152 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:55036, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:54:46,167 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f24080d8c8554a2fb6b31784978a62f3 first=Optional[r05] last=Optional[r8] 2023-03-24 21:54:46,171 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=51 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f24080d8c8554a2fb6b31784978a62f3}] 2023-03-24 21:54:46,200 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f24080d8c8554a2fb6b31784978a62f3 for inclusion in 9329e175b349e4e9722958f8463695ce/family1 2023-03-24 21:54:46,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-24 21:54:46,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:54:46,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:46,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f24080d8c8554a2fb6b31784978a62f3 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jq1hujs4semen54rtusqu3gbfvq86q3mu15djksvbgrau3pqklub9r7hfdjlm4k1/family1/f24080d8c8554a2fb6b31784978a62f3 2023-03-24 21:54:46,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jq1hujs4semen54rtusqu3gbfvq86q3mu15djksvbgrau3pqklub9r7hfdjlm4k1/family1/f24080d8c8554a2fb6b31784978a62f3 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/0b9546ee7c254bd8b6e594039cfc33b2_SeqId_53_ 2023-03-24 21:54:46,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jq1hujs4semen54rtusqu3gbfvq86q3mu15djksvbgrau3pqklub9r7hfdjlm4k1/family1/f24080d8c8554a2fb6b31784978a62f3 into 9329e175b349e4e9722958f8463695ce/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/0b9546ee7c254bd8b6e594039cfc33b2_SeqId_53_ - updating store file list. 2023-03-24 21:54:46,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/0b9546ee7c254bd8b6e594039cfc33b2_SeqId_53_ into 9329e175b349e4e9722958f8463695ce/family1 2023-03-24 21:54:46,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jq1hujs4semen54rtusqu3gbfvq86q3mu15djksvbgrau3pqklub9r7hfdjlm4k1/family1/f24080d8c8554a2fb6b31784978a62f3 into 9329e175b349e4e9722958f8463695ce/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/0b9546ee7c254bd8b6e594039cfc33b2_SeqId_53_) 2023-03-24 21:54:46,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jq1hujs4semen54rtusqu3gbfvq86q3mu15djksvbgrau3pqklub9r7hfdjlm4k1/family1/f24080d8c8554a2fb6b31784978a62f3 2023-03-24 21:54:46,218 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-24 21:54:46,219 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:46,221 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112013676f6ba4e6440fbaf93637088e79f7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112013676f6ba4e6440fbaf93637088e79f7 2023-03-24 21:54:46,222 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511280e59b363b62242b1b78c5bd8a401c17d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511280e59b363b62242b1b78c5bd8a401c17d 2023-03-24 21:54:46,222 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-03-24 21:54:46,222 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-24 21:54:46,223 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:46,223 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:46,223 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:46,223 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-24 21:54:46,223 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:46,224 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-24 21:54:46,225 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 578 msec 2023-03-24 21:54:46,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$12(2454): Client=jenkins//172.31.2.10 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-03-24 21:54:46,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:46,705 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-24 21:54:46,708 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-24 21:54:46,708 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-24 21:54:46,710 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-24 21:54:46,716 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=9329e175b349e4e9722958f8463695ce, REOPEN/MOVE}] 2023-03-24 21:54:46,717 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=9329e175b349e4e9722958f8463695ce, REOPEN/MOVE 2023-03-24 21:54:46,717 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=9329e175b349e4e9722958f8463695ce, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:46,717 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679694886717"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694886717"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694886717"}]},"ts":"1679694886717"} 2023-03-24 21:54:46,719 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 9329e175b349e4e9722958f8463695ce, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:46,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:46,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 9329e175b349e4e9722958f8463695ce, disabling compactions & flushes 2023-03-24 21:54:46,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:46,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:46,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. after waiting 0 ms 2023-03-24 21:54:46,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:46,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-24 21:54:46,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:46,877 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:46,877 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 9329e175b349e4e9722958f8463695ce to self. 2023-03-24 21:54:46,878 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:46,878 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=9329e175b349e4e9722958f8463695ce, regionState=CLOSED 2023-03-24 21:54:46,879 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679694886878"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694886878"}]},"ts":"1679694886878"} 2023-03-24 21:54:46,881 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-03-24 21:54:46,881 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 9329e175b349e4e9722958f8463695ce, server=jenkins-hbase9.apache.org,41807,1679694819103 in 161 msec 2023-03-24 21:54:46,882 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=9329e175b349e4e9722958f8463695ce, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=true 2023-03-24 21:54:47,032 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=9329e175b349e4e9722958f8463695ce, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:47,033 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679694887032"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694887032"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694887032"}]},"ts":"1679694887032"} 2023-03-24 21:54:47,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 9329e175b349e4e9722958f8463695ce, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:47,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:47,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9329e175b349e4e9722958f8463695ce, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:54:47,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:47,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:47,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:47,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:47,190 INFO [StoreOpener-9329e175b349e4e9722958f8463695ce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:47,191 DEBUG [StoreOpener-9329e175b349e4e9722958f8463695ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1 2023-03-24 21:54:47,191 DEBUG [StoreOpener-9329e175b349e4e9722958f8463695ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1 2023-03-24 21:54:47,192 INFO [StoreOpener-9329e175b349e4e9722958f8463695ce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9329e175b349e4e9722958f8463695ce columnFamilyName family1 2023-03-24 21:54:47,197 DEBUG [StoreOpener-9329e175b349e4e9722958f8463695ce-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/0b9546ee7c254bd8b6e594039cfc33b2_SeqId_53_ 2023-03-24 21:54:47,200 DEBUG [StoreOpener-9329e175b349e4e9722958f8463695ce-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/family1/821d4b2e994e451f86c9364a899be68d 2023-03-24 21:54:47,200 INFO [StoreOpener-9329e175b349e4e9722958f8463695ce-1] regionserver.HStore(310): Store=9329e175b349e4e9722958f8463695ce/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:47,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:47,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:47,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 9329e175b349e4e9722958f8463695ce 2023-03-24 21:54:47,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 9329e175b349e4e9722958f8463695ce; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69944333, jitterRate=0.04225178062915802}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:54:47,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:54:47,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce., pid=69, masterSystemTime=1679694887186 2023-03-24 21:54:47,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:47,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:54:47,208 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=9329e175b349e4e9722958f8463695ce, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:47,208 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679694887208"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694887208"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694887208"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694887208"}]},"ts":"1679694887208"} 2023-03-24 21:54:47,211 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-03-24 21:54:47,211 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 9329e175b349e4e9722958f8463695ce, server=jenkins-hbase9.apache.org,41807,1679694819103 in 175 msec 2023-03-24 21:54:47,212 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-03-24 21:54:47,212 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=9329e175b349e4e9722958f8463695ce, REOPEN/MOVE in 495 msec 2023-03-24 21:54:47,214 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-03-24 21:54:47,214 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 503 msec 2023-03-24 21:54:47,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 927 msec 2023-03-24 21:54:47,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-03-24 21:54:47,217 INFO [Listener at localhost/39907] compactions.TestMobCompactor(278): alter status finished 2023-03-24 21:54:47,218 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-24 21:54:47,219 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-24 21:54:47,220 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-24 21:54:47,221 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695487221 2023-03-24 21:54:47,221 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:47,221 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-24 21:54:47,222 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-24 21:54:47,222 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-24 21:54:47,223 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x0558b5c1 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:54:47,227 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3884699e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:54:47,227 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:47,227 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:47,227 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:47,228 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:47,228 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:47,228 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:47,228 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:47,228 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-24 21:54:47,228 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-24 21:54:47,228 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:47,229 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-24 21:54:47,230 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 10 msec 2023-03-24 21:54:47,286 INFO [Listener at localhost/39907] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=257 (was 264), OpenFileDescriptor=654 (was 645) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=369 (was 413), ProcessCount=169 (was 169), AvailableMemoryMB=8086 (was 8115) 2023-03-24 21:54:47,302 INFO [Listener at localhost/39907] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=257, OpenFileDescriptor=654, MaxFileDescriptor=60000, SystemLoadAverage=369, ProcessCount=169, AvailableMemoryMB=8084 2023-03-24 21:54:47,306 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-24 21:54:47,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-03-24 21:54:47,311 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:54:47,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-03-24 21:54:47,312 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:54:47,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-24 21:54:47,314 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:47,314 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de empty. 2023-03-24 21:54:47,315 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:47,315 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-24 21:54:47,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-24 21:54:47,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-24 21:54:47,745 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-03-24 21:54:47,747 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => eaf80714dc731bd492e171a0240209de, NAME => 'testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:54:47,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-24 21:54:48,155 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:48,155 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing eaf80714dc731bd492e171a0240209de, disabling compactions & flushes 2023-03-24 21:54:48,156 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:48,156 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:48,156 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. after waiting 0 ms 2023-03-24 21:54:48,156 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:48,156 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:48,156 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for eaf80714dc731bd492e171a0240209de: 2023-03-24 21:54:48,159 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:54:48,160 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679694888160"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694888160"}]},"ts":"1679694888160"} 2023-03-24 21:54:48,163 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-24 21:54:48,164 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:54:48,164 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694888164"}]},"ts":"1679694888164"} 2023-03-24 21:54:48,166 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-03-24 21:54:48,169 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=eaf80714dc731bd492e171a0240209de, ASSIGN}] 2023-03-24 21:54:48,171 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=eaf80714dc731bd492e171a0240209de, ASSIGN 2023-03-24 21:54:48,174 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=eaf80714dc731bd492e171a0240209de, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:54:48,325 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=eaf80714dc731bd492e171a0240209de, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:48,326 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679694888325"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694888325"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694888325"}]},"ts":"1679694888325"} 2023-03-24 21:54:48,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure eaf80714dc731bd492e171a0240209de, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:48,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-24 21:54:48,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:48,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eaf80714dc731bd492e171a0240209de, NAME => 'testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:54:48,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-03-24 21:54:48,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-03-24 21:54:48,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:48,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:48,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:48,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:48,486 INFO [StoreOpener-eaf80714dc731bd492e171a0240209de-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:48,488 DEBUG [StoreOpener-eaf80714dc731bd492e171a0240209de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1 2023-03-24 21:54:48,488 DEBUG [StoreOpener-eaf80714dc731bd492e171a0240209de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1 2023-03-24 21:54:48,488 INFO [StoreOpener-eaf80714dc731bd492e171a0240209de-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eaf80714dc731bd492e171a0240209de columnFamilyName f1 2023-03-24 21:54:48,490 INFO [StoreOpener-eaf80714dc731bd492e171a0240209de-1] regionserver.HStore(310): Store=eaf80714dc731bd492e171a0240209de/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:48,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:48,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:48,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:48,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:54:48,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened eaf80714dc731bd492e171a0240209de; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64742632, jitterRate=-0.03525960445404053}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:54:48,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for eaf80714dc731bd492e171a0240209de: 2023-03-24 21:54:48,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de., pid=73, masterSystemTime=1679694888479 2023-03-24 21:54:48,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:48,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:48,503 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=eaf80714dc731bd492e171a0240209de, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:48,503 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679694888502"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694888502"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694888502"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694888502"}]},"ts":"1679694888502"} 2023-03-24 21:54:48,505 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-03-24 21:54:48,505 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure eaf80714dc731bd492e171a0240209de, server=jenkins-hbase9.apache.org,41807,1679694819103 in 177 msec 2023-03-24 21:54:48,507 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-03-24 21:54:48,507 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=eaf80714dc731bd492e171a0240209de, ASSIGN in 336 msec 2023-03-24 21:54:48,508 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:54:48,508 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694888508"}]},"ts":"1679694888508"} 2023-03-24 21:54:48,509 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-03-24 21:54:48,511 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:54:48,513 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.2050 sec 2023-03-24 21:54:49,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-24 21:54:49,419 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-03-24 21:54:49,420 DEBUG [Listener at localhost/39907] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-03-24 21:54:49,422 INFO [Listener at localhost/39907] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-24 21:54:49,425 INFO [Listener at localhost/39907] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-03-24 21:54:49,426 INFO [Listener at localhost/39907] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-24 21:54:49,426 INFO [Listener at localhost/39907] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-03-24 21:54:49,432 INFO [Listener at localhost/39907] regionserver.HRegion(2745): Flushing eaf80714dc731bd492e171a0240209de 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-24 21:54:49,445 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324c02590677a20467ca03d06f3603ddecf to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230324c02590677a20467ca03d06f3603ddecf 2023-03-24 21:54:49,851 INFO [Listener at localhost/39907] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/.tmp/f1/231ad397c49744b18401ec1ba370e9d3 2023-03-24 21:54:49,859 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/.tmp/f1/231ad397c49744b18401ec1ba370e9d3 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3 2023-03-24 21:54:49,865 INFO [Listener at localhost/39907] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3, entries=1, sequenceid=5, filesize=4.9 K 2023-03-24 21:54:49,866 INFO [Listener at localhost/39907] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for eaf80714dc731bd492e171a0240209de in 435ms, sequenceid=5, compaction requested=false 2023-03-24 21:54:49,866 DEBUG [Listener at localhost/39907] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-03-24 21:54:49,867 DEBUG [Listener at localhost/39907] regionserver.HRegion(2446): Flush status journal for eaf80714dc731bd492e171a0240209de: 2023-03-24 21:54:49,870 INFO [Listener at localhost/39907] regionserver.HRegion(2745): Flushing eaf80714dc731bd492e171a0240209de 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-24 21:54:49,886 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324dd0ee6339f994ae7bede5e6cec9b1832 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230324dd0ee6339f994ae7bede5e6cec9b1832 2023-03-24 21:54:49,896 INFO [Listener at localhost/39907] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/.tmp/f1/7e893a2d2e38447184361b20a0f76f5c 2023-03-24 21:54:49,902 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/.tmp/f1/7e893a2d2e38447184361b20a0f76f5c as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c 2023-03-24 21:54:49,907 INFO [Listener at localhost/39907] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c, entries=1, sequenceid=9, filesize=4.9 K 2023-03-24 21:54:49,908 INFO [Listener at localhost/39907] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for eaf80714dc731bd492e171a0240209de in 38ms, sequenceid=9, compaction requested=false 2023-03-24 21:54:49,908 DEBUG [Listener at localhost/39907] regionserver.HRegion(2446): Flush status journal for eaf80714dc731bd492e171a0240209de: 2023-03-24 21:54:49,910 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-03-24 21:54:49,912 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-24 21:54:49,912 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-24 21:54:49,913 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:49,913 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695489913 2023-03-24 21:54:49,915 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-24 21:54:49,915 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-03-24 21:54:49,915 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-03-24 21:54:49,916 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x6c0e0f2d to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:54:49,923 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@775d53ab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:54:49,923 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230324 for table='testGetAfterCompaction' and column='f1' 2023-03-24 21:54:49,962 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:49,962 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,013 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,013 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,064 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,064 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,115 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,115 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,166 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,166 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,217 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,217 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,268 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,269 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,335 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:54:50,335 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,335 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3c0f5d0f to 127.0.0.1:54680 2023-03-24 21:54:50,336 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,336 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:50,337 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:54:50,337 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x54f1cde6 to 127.0.0.1:54680 2023-03-24 21:54:50,337 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:50,340 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0558b5c1 to 127.0.0.1:54680 2023-03-24 21:54:50,340 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:50,340 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:54:50,340 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x548b2894 to 127.0.0.1:54680 2023-03-24 21:54:50,340 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:50,340 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5e9c6060 to 127.0.0.1:54680 2023-03-24 21:54:50,340 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:50,340 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:54:50,340 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3f8d6df4 to 127.0.0.1:54680 2023-03-24 21:54:50,340 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:50,341 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:54:50,341 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x692257b5 to 127.0.0.1:54680 2023-03-24 21:54:50,341 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:50,386 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,387 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,437 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,438 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,488 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,489 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,539 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,539 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,591 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,591 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,642 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,642 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,693 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,693 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,743 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,744 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,775 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324f6565903d8db4348acb7e74b64471d79 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230324f6565903d8db4348acb7e74b64471d79 2023-03-24 21:54:50,778 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230324 table='testGetAfterCompaction' column='f1' 2023-03-24 21:54:50,793 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:54:50,794 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:43082, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:54:50,795 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,795 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,804 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:54:50,806 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:54462, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:54:50,817 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230324/f1/c7da54da49bd46b9a2dbabd782a47c79 first=Optional[r1] last=Optional[r2] 2023-03-24 21:54:50,817 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{f1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230324/f1/c7da54da49bd46b9a2dbabd782a47c79}] 2023-03-24 21:54:50,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230324/f1/c7da54da49bd46b9a2dbabd782a47c79 for inclusion in eaf80714dc731bd492e171a0240209de/f1 2023-03-24 21:54:50,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-24 21:54:50,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:54:50,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for eaf80714dc731bd492e171a0240209de: 2023-03-24 21:54:50,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230324/f1/c7da54da49bd46b9a2dbabd782a47c79 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testGetAfterCompaction__5q36cvt4uu5qhb1v6vn8fra4v47ai5fbr462857bg9rkg1077e7nqj7tdv7m0571/f1/c7da54da49bd46b9a2dbabd782a47c79 2023-03-24 21:54:50,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testGetAfterCompaction__5q36cvt4uu5qhb1v6vn8fra4v47ai5fbr462857bg9rkg1077e7nqj7tdv7m0571/f1/c7da54da49bd46b9a2dbabd782a47c79 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/926af8427e4641bdb2144e3cd794b91b_SeqId_12_ 2023-03-24 21:54:50,829 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testGetAfterCompaction__5q36cvt4uu5qhb1v6vn8fra4v47ai5fbr462857bg9rkg1077e7nqj7tdv7m0571/f1/c7da54da49bd46b9a2dbabd782a47c79 into eaf80714dc731bd492e171a0240209de/f1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/926af8427e4641bdb2144e3cd794b91b_SeqId_12_ - updating store file list. 2023-03-24 21:54:50,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/926af8427e4641bdb2144e3cd794b91b_SeqId_12_ into eaf80714dc731bd492e171a0240209de/f1 2023-03-24 21:54:50,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testGetAfterCompaction__5q36cvt4uu5qhb1v6vn8fra4v47ai5fbr462857bg9rkg1077e7nqj7tdv7m0571/f1/c7da54da49bd46b9a2dbabd782a47c79 into eaf80714dc731bd492e171a0240209de/f1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/926af8427e4641bdb2144e3cd794b91b_SeqId_12_) 2023-03-24 21:54:50,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testGetAfterCompaction__5q36cvt4uu5qhb1v6vn8fra4v47ai5fbr462857bg9rkg1077e7nqj7tdv7m0571/f1/c7da54da49bd46b9a2dbabd782a47c79 2023-03-24 21:54:50,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(1912): eaf80714dc731bd492e171a0240209de/f1 is initiating minor compaction 2023-03-24 21:54:50,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@72e40c64; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:50,836 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of eaf80714dc731bd492e171a0240209de/f1 in testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:50,836 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/926af8427e4641bdb2144e3cd794b91b_SeqId_12_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/.tmp, totalSize=10.0 K 2023-03-24 21:54:50,836 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 7e893a2d2e38447184361b20a0f76f5c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-24 21:54:50,836 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 926af8427e4641bdb2144e3cd794b91b_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-24 21:54:50,845 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230324 table='testGetAfterCompaction' column='f1' 2023-03-24 21:54:50,846 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): eaf80714dc731bd492e171a0240209de#f1#compaction#76 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-03-24 21:54:50,846 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,846 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,846 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/926af8427e4641bdb2144e3cd794b91b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679694890769; access_time=1679694889940; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-24 21:54:50,847 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:50,849 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230324c02590677a20467ca03d06f3603ddecf to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230324c02590677a20467ca03d06f3603ddecf 2023-03-24 21:54:50,851 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230324dd0ee6339f994ae7bede5e6cec9b1832 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230324dd0ee6339f994ae7bede5e6cec9b1832 2023-03-24 21:54:50,851 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-03-24 21:54:50,851 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230324' 2023-03-24 21:54:50,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230324 for table='testGetAfterCompaction' and column='f1' 2023-03-24 21:54:50,852 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-03-24 21:54:50,852 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:54:50,859 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-24 21:54:50,861 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 948 msec 2023-03-24 21:54:50,868 DEBUG [Listener at localhost/39907] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-24 21:54:50,873 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:43086, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-24 21:54:50,873 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41807] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:50,873 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-03-24 21:54:50,873 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-24 21:54:50,874 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41807] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-24 21:54:50,874 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41807] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. because compaction request was cancelled 2023-03-24 21:54:50,884 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-24 21:54:50,893 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/.tmp/f1/cb43cef6a4ac459d876a52bfb395166d as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/cb43cef6a4ac459d876a52bfb395166d 2023-03-24 21:54:50,898 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in eaf80714dc731bd492e171a0240209de/f1 of eaf80714dc731bd492e171a0240209de into cb43cef6a4ac459d876a52bfb395166d(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:54:50,899 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for eaf80714dc731bd492e171a0240209de: 2023-03-24 21:54:50,899 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1679694890835; duration=0sec 2023-03-24 21:54:50,899 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:54:50,925 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,926 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889895; access_time=1679694889872; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,926 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/926af8427e4641bdb2144e3cd794b91b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679694890769; access_time=1679694889940; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-24 21:54:50,926 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/cb43cef6a4ac459d876a52bfb395166d; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1679694890883; access_time=1679694890843; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,962 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/926af8427e4641bdb2144e3cd794b91b_SeqId_12_] to archive 2023-03-24 21:54:50,963 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:54:50,965 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/7e893a2d2e38447184361b20a0f76f5c 2023-03-24 21:54:50,966 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/926af8427e4641bdb2144e3cd794b91b_SeqId_12_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/926af8427e4641bdb2144e3cd794b91b_SeqId_12_ 2023-03-24 21:54:50,976 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679694889851; access_time=1679694889435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:50,976 INFO [Listener at localhost/39907] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/cb43cef6a4ac459d876a52bfb395166d; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1679694890883; access_time=1679694890843; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-24 21:54:51,095 INFO [Listener at localhost/39907] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-03-24 21:54:51,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$11(2418): Client=jenkins//172.31.2.10 disable testGetAfterCompaction 2023-03-24 21:54:51,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-03-24 21:54:51,111 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694891111"}]},"ts":"1679694891111"} 2023-03-24 21:54:51,112 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-03-24 21:54:51,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-24 21:54:51,116 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-03-24 21:54:51,117 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=eaf80714dc731bd492e171a0240209de, UNASSIGN}] 2023-03-24 21:54:51,119 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=eaf80714dc731bd492e171a0240209de, UNASSIGN 2023-03-24 21:54:51,119 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=eaf80714dc731bd492e171a0240209de, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:51,119 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679694891119"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694891119"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694891119"}]},"ts":"1679694891119"} 2023-03-24 21:54:51,120 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure eaf80714dc731bd492e171a0240209de, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:51,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-24 21:54:51,272 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:51,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing eaf80714dc731bd492e171a0240209de, disabling compactions & flushes 2023-03-24 21:54:51,272 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:51,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:51,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. after waiting 0 ms 2023-03-24 21:54:51,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:51,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-24 21:54:51,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-24 21:54:51,281 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de. 2023-03-24 21:54:51,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for eaf80714dc731bd492e171a0240209de: 2023-03-24 21:54:51,288 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:51,289 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=eaf80714dc731bd492e171a0240209de, regionState=CLOSED 2023-03-24 21:54:51,289 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679694891289"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694891289"}]},"ts":"1679694891289"} 2023-03-24 21:54:51,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-03-24 21:54:51,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure eaf80714dc731bd492e171a0240209de, server=jenkins-hbase9.apache.org,41807,1679694819103 in 173 msec 2023-03-24 21:54:51,297 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-03-24 21:54:51,297 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=eaf80714dc731bd492e171a0240209de, UNASSIGN in 178 msec 2023-03-24 21:54:51,298 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694891298"}]},"ts":"1679694891298"} 2023-03-24 21:54:51,299 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-03-24 21:54:51,300 INFO [PEWorker-4] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-03-24 21:54:51,302 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 199 msec 2023-03-24 21:54:51,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-24 21:54:51,415 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-03-24 21:54:51,420 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$5(2228): Client=jenkins//172.31.2.10 delete testGetAfterCompaction 2023-03-24 21:54:51,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-03-24 21:54:51,428 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-24 21:54:51,430 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-24 21:54:51,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-24 21:54:51,433 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:51,435 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1, FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/recovered.edits] 2023-03-24 21:54:51,439 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/231ad397c49744b18401ec1ba370e9d3 2023-03-24 21:54:51,441 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/cb43cef6a4ac459d876a52bfb395166d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/f1/cb43cef6a4ac459d876a52bfb395166d 2023-03-24 21:54:51,443 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/recovered.edits/15.seqid to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de/recovered.edits/15.seqid 2023-03-24 21:54:51,444 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testGetAfterCompaction/eaf80714dc731bd492e171a0240209de 2023-03-24 21:54:51,444 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-24 21:54:51,445 DEBUG [PEWorker-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-24 21:54:51,445 DEBUG [PEWorker-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-03-24 21:54:51,449 DEBUG [PEWorker-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230324f6565903d8db4348acb7e74b64471d79 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230324f6565903d8db4348acb7e74b64471d79 2023-03-24 21:54:51,449 DEBUG [PEWorker-2] backup.HFileArchiver(596): Deleted hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-24 21:54:51,451 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-24 21:54:51,476 WARN [PEWorker-2] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-03-24 21:54:51,478 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-03-24 21:54:51,479 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-24 21:54:51,479 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-03-24 21:54:51,480 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679694891479"}]},"ts":"9223372036854775807"} 2023-03-24 21:54:51,481 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-03-24 21:54:51,481 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => eaf80714dc731bd492e171a0240209de, NAME => 'testGetAfterCompaction,,1679694887306.eaf80714dc731bd492e171a0240209de.', STARTKEY => '', ENDKEY => ''}] 2023-03-24 21:54:51,481 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-03-24 21:54:51,481 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1679694891481"}]},"ts":"9223372036854775807"} 2023-03-24 21:54:51,483 INFO [PEWorker-2] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-03-24 21:54:51,485 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-24 21:54:51,486 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 64 msec 2023-03-24 21:54:51,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-24 21:54:51,532 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-03-24 21:54:51,548 INFO [Listener at localhost/39907] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=243 (was 257), OpenFileDescriptor=608 (was 654), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=369 (was 369), ProcessCount=169 (was 169), AvailableMemoryMB=8008 (was 8084) 2023-03-24 21:54:51,564 INFO [Listener at localhost/39907] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=243, OpenFileDescriptor=608, MaxFileDescriptor=60000, SystemLoadAverage=369, ProcessCount=169, AvailableMemoryMB=8008 2023-03-24 21:54:51,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-24 21:54:51,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-03-24 21:54:51,569 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:54:51,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-03-24 21:54:51,569 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:54:51,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-24 21:54:51,572 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:54:51,572 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:54:51,572 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2 2023-03-24 21:54:51,572 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2 empty. 2023-03-24 21:54:51,573 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d empty. 2023-03-24 21:54:51,573 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2 empty. 2023-03-24 21:54:51,573 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2 2023-03-24 21:54:51,573 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:54:51,573 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:54:51,573 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-03-24 21:54:51,582 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-24 21:54:51,585 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => 80c7f51f73f3bd5497dc15e88fadefa2, NAME => 'testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:54:51,586 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => dd7d45dee3d2e56a7d8a4ae56820f04d, NAME => 'testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:54:51,586 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => db6e510d903e066fe70066967a0093e2, NAME => 'testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:54:51,618 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:51,618 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing 80c7f51f73f3bd5497dc15e88fadefa2, disabling compactions & flushes 2023-03-24 21:54:51,618 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:51,618 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:51,618 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. after waiting 0 ms 2023-03-24 21:54:51,618 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:51,618 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:51,618 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:54:51,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-24 21:54:51,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-24 21:54:52,009 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:52,009 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing db6e510d903e066fe70066967a0093e2, disabling compactions & flushes 2023-03-24 21:54:52,009 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:52,009 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:52,009 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. after waiting 0 ms 2023-03-24 21:54:52,009 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:52,009 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:52,009 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:52,009 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:54:52,010 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing dd7d45dee3d2e56a7d8a4ae56820f04d, disabling compactions & flushes 2023-03-24 21:54:52,010 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:52,010 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:52,010 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. after waiting 0 ms 2023-03-24 21:54:52,010 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:52,010 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:52,010 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:54:52,012 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:54:52,013 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694892013"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694892013"}]},"ts":"1679694892013"} 2023-03-24 21:54:52,013 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694892013"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694892013"}]},"ts":"1679694892013"} 2023-03-24 21:54:52,013 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694892013"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694892013"}]},"ts":"1679694892013"} 2023-03-24 21:54:52,015 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-24 21:54:52,016 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:54:52,016 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694892016"}]},"ts":"1679694892016"} 2023-03-24 21:54:52,017 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-24 21:54:52,024 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=80c7f51f73f3bd5497dc15e88fadefa2, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd7d45dee3d2e56a7d8a4ae56820f04d, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=db6e510d903e066fe70066967a0093e2, ASSIGN}] 2023-03-24 21:54:52,026 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd7d45dee3d2e56a7d8a4ae56820f04d, ASSIGN 2023-03-24 21:54:52,026 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=db6e510d903e066fe70066967a0093e2, ASSIGN 2023-03-24 21:54:52,026 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=80c7f51f73f3bd5497dc15e88fadefa2, ASSIGN 2023-03-24 21:54:52,027 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd7d45dee3d2e56a7d8a4ae56820f04d, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:54:52,036 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=80c7f51f73f3bd5497dc15e88fadefa2, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:54:52,036 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=db6e510d903e066fe70066967a0093e2, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:54:52,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-24 21:54:52,179 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=db6e510d903e066fe70066967a0093e2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:52,179 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=80c7f51f73f3bd5497dc15e88fadefa2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:52,179 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=dd7d45dee3d2e56a7d8a4ae56820f04d, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:52,179 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694892179"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694892179"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694892179"}]},"ts":"1679694892179"} 2023-03-24 21:54:52,179 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694892179"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694892179"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694892179"}]},"ts":"1679694892179"} 2023-03-24 21:54:52,179 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694892179"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694892179"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694892179"}]},"ts":"1679694892179"} 2023-03-24 21:54:52,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=82, state=RUNNABLE; OpenRegionProcedure db6e510d903e066fe70066967a0093e2, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:52,181 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=80, state=RUNNABLE; OpenRegionProcedure 80c7f51f73f3bd5497dc15e88fadefa2, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:52,183 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=81, state=RUNNABLE; OpenRegionProcedure dd7d45dee3d2e56a7d8a4ae56820f04d, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:54:52,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:52,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 80c7f51f73f3bd5497dc15e88fadefa2, NAME => 'testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.', STARTKEY => '', ENDKEY => '1'} 2023-03-24 21:54:52,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:54:52,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:52,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:54:52,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:54:52,338 INFO [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:54:52,340 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1 2023-03-24 21:54:52,340 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1 2023-03-24 21:54:52,340 INFO [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80c7f51f73f3bd5497dc15e88fadefa2 columnFamilyName family1 2023-03-24 21:54:52,342 INFO [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(310): Store=80c7f51f73f3bd5497dc15e88fadefa2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:52,342 INFO [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:54:52,343 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2 2023-03-24 21:54:52,343 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2 2023-03-24 21:54:52,344 INFO [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80c7f51f73f3bd5497dc15e88fadefa2 columnFamilyName family2 2023-03-24 21:54:52,345 INFO [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(310): Store=80c7f51f73f3bd5497dc15e88fadefa2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:52,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:54:52,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:54:52,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:54:52,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:54:52,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:54:52,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 80c7f51f73f3bd5497dc15e88fadefa2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72681294, jitterRate=0.0830356776714325}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:54:52,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:54:52,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2., pid=84, masterSystemTime=1679694892333 2023-03-24 21:54:52,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:52,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:52,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:52,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dd7d45dee3d2e56a7d8a4ae56820f04d, NAME => 'testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.', STARTKEY => '1', ENDKEY => '2'} 2023-03-24 21:54:52,359 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=80c7f51f73f3bd5497dc15e88fadefa2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:52,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:54:52,359 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694892359"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694892359"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694892359"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694892359"}]},"ts":"1679694892359"} 2023-03-24 21:54:52,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:52,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:54:52,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:54:52,360 INFO [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:54:52,362 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1 2023-03-24 21:54:52,362 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1 2023-03-24 21:54:52,362 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=80 2023-03-24 21:54:52,362 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=80, state=SUCCESS; OpenRegionProcedure 80c7f51f73f3bd5497dc15e88fadefa2, server=jenkins-hbase9.apache.org,41807,1679694819103 in 179 msec 2023-03-24 21:54:52,362 INFO [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd7d45dee3d2e56a7d8a4ae56820f04d columnFamilyName family1 2023-03-24 21:54:52,363 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=80c7f51f73f3bd5497dc15e88fadefa2, ASSIGN in 341 msec 2023-03-24 21:54:52,363 INFO [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(310): Store=dd7d45dee3d2e56a7d8a4ae56820f04d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:52,364 INFO [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:54:52,365 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2 2023-03-24 21:54:52,365 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2 2023-03-24 21:54:52,365 INFO [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd7d45dee3d2e56a7d8a4ae56820f04d columnFamilyName family2 2023-03-24 21:54:52,366 INFO [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(310): Store=dd7d45dee3d2e56a7d8a4ae56820f04d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:52,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:54:52,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:54:52,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:54:52,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:54:52,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:54:52,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened dd7d45dee3d2e56a7d8a4ae56820f04d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61734130, jitterRate=-0.08008977770805359}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:54:52,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:54:52,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d., pid=85, masterSystemTime=1679694892333 2023-03-24 21:54:52,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:52,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:52,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:52,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => db6e510d903e066fe70066967a0093e2, NAME => 'testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.', STARTKEY => '2', ENDKEY => ''} 2023-03-24 21:54:52,375 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=dd7d45dee3d2e56a7d8a4ae56820f04d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:52,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink db6e510d903e066fe70066967a0093e2 2023-03-24 21:54:52,376 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694892375"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694892375"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694892375"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694892375"}]},"ts":"1679694892375"} 2023-03-24 21:54:52,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:54:52,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for db6e510d903e066fe70066967a0093e2 2023-03-24 21:54:52,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for db6e510d903e066fe70066967a0093e2 2023-03-24 21:54:52,377 INFO [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region db6e510d903e066fe70066967a0093e2 2023-03-24 21:54:52,381 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1 2023-03-24 21:54:52,381 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1 2023-03-24 21:54:52,382 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=81 2023-03-24 21:54:52,382 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=81, state=SUCCESS; OpenRegionProcedure dd7d45dee3d2e56a7d8a4ae56820f04d, server=jenkins-hbase9.apache.org,41807,1679694819103 in 195 msec 2023-03-24 21:54:52,382 INFO [StoreOpener-db6e510d903e066fe70066967a0093e2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db6e510d903e066fe70066967a0093e2 columnFamilyName family1 2023-03-24 21:54:52,383 INFO [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(310): Store=db6e510d903e066fe70066967a0093e2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:52,383 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd7d45dee3d2e56a7d8a4ae56820f04d, ASSIGN in 361 msec 2023-03-24 21:54:52,383 INFO [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region db6e510d903e066fe70066967a0093e2 2023-03-24 21:54:52,385 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2 2023-03-24 21:54:52,385 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2 2023-03-24 21:54:52,385 INFO [StoreOpener-db6e510d903e066fe70066967a0093e2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db6e510d903e066fe70066967a0093e2 columnFamilyName family2 2023-03-24 21:54:52,386 INFO [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(310): Store=db6e510d903e066fe70066967a0093e2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:54:52,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2 2023-03-24 21:54:52,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2 2023-03-24 21:54:52,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:54:52,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for db6e510d903e066fe70066967a0093e2 2023-03-24 21:54:52,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:54:52,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened db6e510d903e066fe70066967a0093e2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74298881, jitterRate=0.10713960230350494}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:54:52,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:54:52,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2., pid=83, masterSystemTime=1679694892333 2023-03-24 21:54:52,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:52,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:52,396 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=db6e510d903e066fe70066967a0093e2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:52,396 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694892396"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694892396"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694892396"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694892396"}]},"ts":"1679694892396"} 2023-03-24 21:54:52,399 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=82 2023-03-24 21:54:52,399 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=82, state=SUCCESS; OpenRegionProcedure db6e510d903e066fe70066967a0093e2, server=jenkins-hbase9.apache.org,41807,1679694819103 in 217 msec 2023-03-24 21:54:52,400 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=79 2023-03-24 21:54:52,400 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=db6e510d903e066fe70066967a0093e2, ASSIGN in 375 msec 2023-03-24 21:54:52,400 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:54:52,401 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694892401"}]},"ts":"1679694892401"} 2023-03-24 21:54:52,402 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-24 21:54:52,404 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:54:52,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 837 msec 2023-03-24 21:54:52,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-24 21:54:52,675 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-03-24 21:54:52,684 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:54:52,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:54:52,686 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:54:52,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:52,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-24 21:54:52,692 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-24 21:54:52,692 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:52,692 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-24 21:54:52,692 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-24 21:54:52,693 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:52,693 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:52,694 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:52,694 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:52,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:52,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:52,694 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:52,694 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:52,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:52,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:52,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:52,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:52,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-24 21:54:52,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-24 21:54:52,696 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-24 21:54:52,696 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:52,696 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-24 21:54:52,696 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:52,696 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. started... 2023-03-24 21:54:52,696 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:54:52,696 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:52,696 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. started... 2023-03-24 21:54:52,697 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dd7d45dee3d2e56a7d8a4ae56820f04d 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-24 21:54:52,697 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 80c7f51f73f3bd5497dc15e88fadefa2 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-24 21:54:52,696 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:52,698 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. started... 2023-03-24 21:54:52,696 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:52,698 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing db6e510d903e066fe70066967a0093e2 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-24 21:54:52,807 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324dfed51c8c5474a239a39070aba3b8b49 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324dfed51c8c5474a239a39070aba3b8b49 2023-03-24 21:54:52,811 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324ea6e832756384abbbdebb994807e4349 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324ea6e832756384abbbdebb994807e4349 2023-03-24 21:54:52,828 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324d7fcdeeeaa884fb399b7825fb7a06335 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324d7fcdeeeaa884fb399b7825fb7a06335 2023-03-24 21:54:52,838 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family1/8dbaeb15a2174a16b5567bf58b9bbecd 2023-03-24 21:54:52,845 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-24 21:54:52,876 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family1/cd41d6bc7625452d8cd50d0509b4a453 2023-03-24 21:54:52,879 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family1/d27247c35aab4a5e95522f4b4082ee48 2023-03-24 21:54:52,917 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324ee6b06f3c7fd4a228457033777494a55 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324ee6b06f3c7fd4a228457033777494a55 2023-03-24 21:54:52,999 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family2/9430792f9a7c45b2b628cf070eabbb41 2023-03-24 21:54:53,007 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family1/8dbaeb15a2174a16b5567bf58b9bbecd as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/8dbaeb15a2174a16b5567bf58b9bbecd 2023-03-24 21:54:53,007 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324e32db202e5394db1980382e90a3489c3 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230324e32db202e5394db1980382e90a3489c3 2023-03-24 21:54:53,016 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/8dbaeb15a2174a16b5567bf58b9bbecd, entries=4, sequenceid=5, filesize=5.4 K 2023-03-24 21:54:53,018 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family2/9430792f9a7c45b2b628cf070eabbb41 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9430792f9a7c45b2b628cf070eabbb41 2023-03-24 21:54:53,024 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9430792f9a7c45b2b628cf070eabbb41, entries=2, sequenceid=5, filesize=5.1 K 2023-03-24 21:54:53,026 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for db6e510d903e066fe70066967a0093e2 in 328ms, sequenceid=5, compaction requested=false 2023-03-24 21:54:53,027 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:54:53,027 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:53,027 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:54:53,367 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324ae6b4dfca433486082c9e1f7e3e66627 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230324ae6b4dfca433486082c9e1f7e3e66627 2023-03-24 21:54:53,441 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family2/b0f3a1484555406892bf1e9f9963292d 2023-03-24 21:54:53,446 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family1/cd41d6bc7625452d8cd50d0509b4a453 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/cd41d6bc7625452d8cd50d0509b4a453 2023-03-24 21:54:53,452 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/cd41d6bc7625452d8cd50d0509b4a453, entries=4, sequenceid=5, filesize=5.4 K 2023-03-24 21:54:53,455 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family2/b0f3a1484555406892bf1e9f9963292d as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/b0f3a1484555406892bf1e9f9963292d 2023-03-24 21:54:53,459 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/b0f3a1484555406892bf1e9f9963292d, entries=2, sequenceid=5, filesize=5.1 K 2023-03-24 21:54:53,460 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 80c7f51f73f3bd5497dc15e88fadefa2 in 763ms, sequenceid=5, compaction requested=false 2023-03-24 21:54:53,460 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:54:53,460 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:53,460 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:54:53,773 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family2/265c8226a3a1414583f189917e8a53c7 2023-03-24 21:54:53,778 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family1/d27247c35aab4a5e95522f4b4082ee48 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d27247c35aab4a5e95522f4b4082ee48 2023-03-24 21:54:53,787 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d27247c35aab4a5e95522f4b4082ee48, entries=4, sequenceid=5, filesize=5.4 K 2023-03-24 21:54:53,790 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family2/265c8226a3a1414583f189917e8a53c7 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/265c8226a3a1414583f189917e8a53c7 2023-03-24 21:54:53,795 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/265c8226a3a1414583f189917e8a53c7, entries=2, sequenceid=5, filesize=5.1 K 2023-03-24 21:54:53,796 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for dd7d45dee3d2e56a7d8a4ae56820f04d in 1099ms, sequenceid=5, compaction requested=false 2023-03-24 21:54:53,796 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:54:53,796 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:53,797 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:54:53,797 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:54:53,797 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,797 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-24 21:54:53,797 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-24 21:54:53,799 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,799 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:53,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:53,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:53,799 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:53,799 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:53,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:53,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:53,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:53,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:53,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-24 21:54:53,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ba1a917[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:53,802 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-24 21:54:53,802 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:53,804 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:53,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:53,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:53,805 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-24 21:54:53,805 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-24 21:54:53,805 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:53,805 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,805 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:53,806 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-24 21:54:53,806 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:53,806 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-24 21:54:53,806 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:53,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:53,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:53,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:53,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:53,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:53,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:53,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:53,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-24 21:54:53,810 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-24 21:54:53,810 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:53,810 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:53,810 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-24 21:54:53,810 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:53,813 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:53,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:53,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:53,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:53,814 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:53,814 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:53,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:53,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:53,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:53,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:53,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:53,815 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:53,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:53,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:53,817 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:53,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:53,820 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:53,820 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:53,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:53,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:53,820 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,820 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:53,821 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:53,821 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,821 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:53,821 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:53,821 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:53,821 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:53,821 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-24 21:54:53,821 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:53,821 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-24 21:54:53,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:53,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:53,821 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:53,822 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:53,922 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:53,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:53,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:54:53,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:54:53,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:54:53,927 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:53,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:53,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-24 21:54:53,930 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-24 21:54:53,930 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:53,930 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-24 21:54:53,930 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-24 21:54:53,930 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:53,930 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:53,932 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:53,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:53,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:53,932 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:53,932 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:53,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:53,933 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:53,933 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:53,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:53,933 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-24 21:54:53,933 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-24 21:54:53,933 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-24 21:54:53,933 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-24 21:54:53,934 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:53,934 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-24 21:54:53,934 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:53,934 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. started... 2023-03-24 21:54:53,934 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:53,934 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. started... 2023-03-24 21:54:53,934 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dd7d45dee3d2e56a7d8a4ae56820f04d 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-24 21:54:53,934 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing db6e510d903e066fe70066967a0093e2 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-24 21:54:53,939 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:54:53,939 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:53,939 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:53,939 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. started... 2023-03-24 21:54:53,940 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 80c7f51f73f3bd5497dc15e88fadefa2 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-24 21:54:53,974 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324d7da3a5c947a4742b6e70403b0b8915b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324d7da3a5c947a4742b6e70403b0b8915b 2023-03-24 21:54:53,974 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032408ac34e9f9cf436e9e176b1f391cb5cf to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032408ac34e9f9cf436e9e176b1f391cb5cf 2023-03-24 21:54:54,377 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032438af3f7a90284a219128aa718c8b109e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032438af3f7a90284a219128aa718c8b109e 2023-03-24 21:54:54,380 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family1/3a255a9bd6b34f9e82ea9b25a9f3431b 2023-03-24 21:54:54,380 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family1/0ade5f95824844f1bdb3bf6657afc66c 2023-03-24 21:54:54,396 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family1/57f179ca64e64316ba7cd9ca99b22557 2023-03-24 21:54:54,420 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303248f0be98635a74e8cb762dd3ac9d9dad1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303248f0be98635a74e8cb762dd3ac9d9dad1 2023-03-24 21:54:54,422 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324771fcbddd6164a379354a7c114971988 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324771fcbddd6164a379354a7c114971988 2023-03-24 21:54:54,437 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032448d400f1b8b64d3b8f83bec372eef9b3 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032448d400f1b8b64d3b8f83bec372eef9b3 2023-03-24 21:54:54,442 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family2/aad477eb45dc4b8fbb0b9c5cc5ab0982 2023-03-24 21:54:54,447 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family1/3a255a9bd6b34f9e82ea9b25a9f3431b as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/3a255a9bd6b34f9e82ea9b25a9f3431b 2023-03-24 21:54:54,451 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family2/7e5ea937345a494e87b1c26c366f8cd7 2023-03-24 21:54:54,452 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/3a255a9bd6b34f9e82ea9b25a9f3431b, entries=4, sequenceid=9, filesize=5.4 K 2023-03-24 21:54:54,452 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family2/0eb237e6e4534356b0c3eb93951ca94b 2023-03-24 21:54:54,453 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family2/aad477eb45dc4b8fbb0b9c5cc5ab0982 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/aad477eb45dc4b8fbb0b9c5cc5ab0982 2023-03-24 21:54:54,457 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family1/57f179ca64e64316ba7cd9ca99b22557 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/57f179ca64e64316ba7cd9ca99b22557 2023-03-24 21:54:54,458 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family1/0ade5f95824844f1bdb3bf6657afc66c as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/0ade5f95824844f1bdb3bf6657afc66c 2023-03-24 21:54:54,458 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/aad477eb45dc4b8fbb0b9c5cc5ab0982, entries=2, sequenceid=9, filesize=5.1 K 2023-03-24 21:54:54,460 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for dd7d45dee3d2e56a7d8a4ae56820f04d in 526ms, sequenceid=9, compaction requested=false 2023-03-24 21:54:54,460 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:54:54,460 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:54,460 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:54:54,464 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/57f179ca64e64316ba7cd9ca99b22557, entries=4, sequenceid=9, filesize=5.4 K 2023-03-24 21:54:54,465 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family2/7e5ea937345a494e87b1c26c366f8cd7 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/7e5ea937345a494e87b1c26c366f8cd7 2023-03-24 21:54:54,468 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/0ade5f95824844f1bdb3bf6657afc66c, entries=4, sequenceid=9, filesize=5.4 K 2023-03-24 21:54:54,469 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family2/0eb237e6e4534356b0c3eb93951ca94b as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/0eb237e6e4534356b0c3eb93951ca94b 2023-03-24 21:54:54,470 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/7e5ea937345a494e87b1c26c366f8cd7, entries=2, sequenceid=9, filesize=5.1 K 2023-03-24 21:54:54,471 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 80c7f51f73f3bd5497dc15e88fadefa2 in 531ms, sequenceid=9, compaction requested=false 2023-03-24 21:54:54,471 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:54:54,471 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:54,471 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:54:54,477 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/0eb237e6e4534356b0c3eb93951ca94b, entries=2, sequenceid=9, filesize=5.1 K 2023-03-24 21:54:54,483 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for db6e510d903e066fe70066967a0093e2 in 549ms, sequenceid=9, compaction requested=false 2023-03-24 21:54:54,483 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:54:54,483 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:54,483 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:54:54,483 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:54:54,483 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,483 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-24 21:54:54,483 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-24 21:54:54,487 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:54,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:54,487 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:54,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:54,488 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:54,488 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:54,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:54,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:54,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:54,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-24 21:54:54,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c17a278[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:54,490 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-24 21:54:54,490 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:54,492 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:54,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:54,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:54,492 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-24 21:54:54,492 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-24 21:54:54,492 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:54,493 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,493 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:54,494 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-24 21:54:54,494 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,494 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:54,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:54,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:54,494 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-24 21:54:54,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:54,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:54,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:54,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:54,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:54,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:54,512 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-24 21:54:54,512 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-24 21:54:54,512 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:54,512 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:54,512 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-24 21:54:54,512 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:54,514 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:54,514 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:54,514 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:54,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:54,515 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:54,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:54,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:54,518 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:54,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:54,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:54,521 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:54,521 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:54,521 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,521 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:54,521 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:54,521 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:54,521 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-24 21:54:54,521 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:54,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:54,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:54,521 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,521 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:54,521 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:54,521 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:54,521 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:54,521 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:54,522 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-24 21:54:54,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:54,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:54,622 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:54,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:54,625 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:54:54,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:54:54,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:54:54,641 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:54,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:54,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-24 21:54:54,647 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-24 21:54:54,647 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:54,647 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-24 21:54:54,647 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-24 21:54:54,647 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:54,647 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:54,651 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,651 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:54,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:54,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:54,652 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:54,652 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:54,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:54,652 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:54,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:54,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:54,653 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-24 21:54:54,662 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-24 21:54:54,663 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-24 21:54:54,663 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-24 21:54:54,663 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:54,663 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-24 21:54:54,672 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:54,672 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. started... 2023-03-24 21:54:54,672 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dd7d45dee3d2e56a7d8a4ae56820f04d 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-24 21:54:54,673 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:54,673 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. started... 2023-03-24 21:54:54,673 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing db6e510d903e066fe70066967a0093e2 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-24 21:54:54,673 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:54:54,673 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:54,674 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:54,674 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. started... 2023-03-24 21:54:54,674 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 80c7f51f73f3bd5497dc15e88fadefa2 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-24 21:54:55,133 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324d0c5015a0307438da385076af4d9d54f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324d0c5015a0307438da385076af4d9d54f 2023-03-24 21:54:55,139 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324ab93dd4f06c644be89323eed2735e5f0 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324ab93dd4f06c644be89323eed2735e5f0 2023-03-24 21:54:55,141 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family1/00999c369b14425ea335da2d7d948d08 2023-03-24 21:54:55,145 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303242e179998f2e24897b7653543ea105766 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303242e179998f2e24897b7653543ea105766 2023-03-24 21:54:55,161 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303245040c195a5574df4a60c60e92e90ed3a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303245040c195a5574df4a60c60e92e90ed3a 2023-03-24 21:54:55,166 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family2/a87ee0a5f4f1401a967dae27a4650f1a 2023-03-24 21:54:55,171 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family1/00999c369b14425ea335da2d7d948d08 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/00999c369b14425ea335da2d7d948d08 2023-03-24 21:54:55,175 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/00999c369b14425ea335da2d7d948d08, entries=4, sequenceid=13, filesize=5.4 K 2023-03-24 21:54:55,176 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family2/a87ee0a5f4f1401a967dae27a4650f1a as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/a87ee0a5f4f1401a967dae27a4650f1a 2023-03-24 21:54:55,180 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/a87ee0a5f4f1401a967dae27a4650f1a, entries=2, sequenceid=13, filesize=5.1 K 2023-03-24 21:54:55,180 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 80c7f51f73f3bd5497dc15e88fadefa2 in 506ms, sequenceid=13, compaction requested=true 2023-03-24 21:54:55,181 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:54:55,181 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:55,181 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:54:55,546 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family1/c2c3a5c11515473ba6a2ac8f75a53ddb 2023-03-24 21:54:55,553 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family1/d9d7de1995344cbab674aca5eed6eee7 2023-03-24 21:54:55,979 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032436c0ae60afb848de99a9ba12d53edbbe to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032436c0ae60afb848de99a9ba12d53edbbe 2023-03-24 21:54:55,986 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324ee47c7143e3548a0946b7998233c5cb3 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230324ee47c7143e3548a0946b7998233c5cb3 2023-03-24 21:54:56,008 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family2/ff2402da2b7244269d37afebd5825ed5 2023-03-24 21:54:56,013 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family1/d9d7de1995344cbab674aca5eed6eee7 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d9d7de1995344cbab674aca5eed6eee7 2023-03-24 21:54:56,017 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d9d7de1995344cbab674aca5eed6eee7, entries=4, sequenceid=13, filesize=5.4 K 2023-03-24 21:54:56,019 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family2/ff2402da2b7244269d37afebd5825ed5 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/ff2402da2b7244269d37afebd5825ed5 2023-03-24 21:54:56,024 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/ff2402da2b7244269d37afebd5825ed5, entries=2, sequenceid=13, filesize=5.1 K 2023-03-24 21:54:56,025 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for dd7d45dee3d2e56a7d8a4ae56820f04d in 1353ms, sequenceid=13, compaction requested=true 2023-03-24 21:54:56,025 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:54:56,025 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:56,025 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:54:56,395 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family2/6b11f68343084dc3bc8a631c212cf15c 2023-03-24 21:54:56,400 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family1/c2c3a5c11515473ba6a2ac8f75a53ddb as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c2c3a5c11515473ba6a2ac8f75a53ddb 2023-03-24 21:54:56,405 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c2c3a5c11515473ba6a2ac8f75a53ddb, entries=4, sequenceid=13, filesize=5.4 K 2023-03-24 21:54:56,405 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family2/6b11f68343084dc3bc8a631c212cf15c as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/6b11f68343084dc3bc8a631c212cf15c 2023-03-24 21:54:56,411 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/6b11f68343084dc3bc8a631c212cf15c, entries=2, sequenceid=13, filesize=5.1 K 2023-03-24 21:54:56,412 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for db6e510d903e066fe70066967a0093e2 in 1739ms, sequenceid=13, compaction requested=true 2023-03-24 21:54:56,412 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:54:56,412 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:56,412 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:54:56,413 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:54:56,413 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,413 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-24 21:54:56,413 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-24 21:54:56,414 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,414 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:56,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:56,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:56,415 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:56,415 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:56,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:56,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:56,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:56,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:56,416 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-24 21:54:56,416 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47ec7070[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:56,416 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-24 21:54:56,416 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:56,419 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:56,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:56,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:56,419 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-24 21:54:56,419 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-24 21:54:56,419 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,419 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:56,419 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:56,424 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:56,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:56,425 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-24 21:54:56,425 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:56,425 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-24 21:54:56,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:56,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:56,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:56,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:56,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:56,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:56,428 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-24 21:54:56,428 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-24 21:54:56,428 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:56,428 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:56,428 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-24 21:54:56,428 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:56,431 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:56,431 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:56,431 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:56,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:56,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:56,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:56,432 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:56,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:56,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:56,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:56,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:56,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:56,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,433 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:56,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:56,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,438 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:56,438 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,438 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:56,438 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:56,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-24 21:54:56,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:56,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:56,438 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:56,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:56,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-24 21:54:56,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:56,438 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,438 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:56,438 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:56,438 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:56,438 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:56,438 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:56,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:56,439 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:56,539 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:56,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:56,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:54:56,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:54:56,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:54:56,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:56,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:56,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-24 21:54:56,555 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-24 21:54:56,555 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:56,555 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-24 21:54:56,555 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-24 21:54:56,555 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:56,556 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:56,558 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,558 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:56,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:56,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:56,558 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:56,558 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:56,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:56,559 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:56,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:56,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:56,559 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-24 21:54:56,559 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-24 21:54:56,559 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-24 21:54:56,559 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-24 21:54:56,560 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:56,560 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-24 21:54:56,560 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:56,560 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. started... 2023-03-24 21:54:56,560 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dd7d45dee3d2e56a7d8a4ae56820f04d 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-24 21:54:56,561 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:56,561 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. started... 2023-03-24 21:54:56,561 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:54:56,561 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:56,561 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing db6e510d903e066fe70066967a0093e2 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-24 21:54:56,561 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:56,561 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. started... 2023-03-24 21:54:56,561 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 80c7f51f73f3bd5497dc15e88fadefa2 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-24 21:54:56,619 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303249102367c04aa4f6681d55ff092535183 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303249102367c04aa4f6681d55ff092535183 2023-03-24 21:54:56,636 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303240938f5625b8f4728ae55dff1564edffb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303240938f5625b8f4728ae55dff1564edffb 2023-03-24 21:54:56,649 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family1/502a23eefb564b108fd0804dd81ccc60 2023-03-24 21:54:56,672 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303240a18c957ac64464aa452d87984ecd741 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303240a18c957ac64464aa452d87984ecd741 2023-03-24 21:54:56,782 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-24 21:54:57,037 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324af079d2fd1914f4fafbcb41bd9adc5b1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324af079d2fd1914f4fafbcb41bd9adc5b1 2023-03-24 21:54:57,050 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family1/79acd90194484e429e64a28f2b3d5703 2023-03-24 21:54:57,083 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family2/73842ebef8cc4e218b9f1e78ba7eead2 2023-03-24 21:54:57,087 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family1/502a23eefb564b108fd0804dd81ccc60 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/502a23eefb564b108fd0804dd81ccc60 2023-03-24 21:54:57,091 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/502a23eefb564b108fd0804dd81ccc60, entries=4, sequenceid=17, filesize=5.4 K 2023-03-24 21:54:57,092 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family2/73842ebef8cc4e218b9f1e78ba7eead2 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/73842ebef8cc4e218b9f1e78ba7eead2 2023-03-24 21:54:57,097 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/73842ebef8cc4e218b9f1e78ba7eead2, entries=2, sequenceid=17, filesize=5.1 K 2023-03-24 21:54:57,097 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for dd7d45dee3d2e56a7d8a4ae56820f04d in 537ms, sequenceid=17, compaction requested=true 2023-03-24 21:54:57,098 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:54:57,098 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:57,098 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:54:57,444 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family1/343affb19b444075983c87a317f71c54 2023-03-24 21:54:57,461 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032434f646a8dbf04661a5b61817f4fadd69 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032434f646a8dbf04661a5b61817f4fadd69 2023-03-24 21:54:57,470 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032435e7af8aff134bc7978ad6fb7cd6c2be to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032435e7af8aff134bc7978ad6fb7cd6c2be 2023-03-24 21:54:57,871 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family2/0beec99d01ea409ab5eb29b007b9a14f 2023-03-24 21:54:57,876 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family1/343affb19b444075983c87a317f71c54 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/343affb19b444075983c87a317f71c54 2023-03-24 21:54:57,876 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family2/9f76b2a9f280438bb6a623955f71adae 2023-03-24 21:54:57,880 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/343affb19b444075983c87a317f71c54, entries=4, sequenceid=17, filesize=5.4 K 2023-03-24 21:54:57,881 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family2/0beec99d01ea409ab5eb29b007b9a14f as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/0beec99d01ea409ab5eb29b007b9a14f 2023-03-24 21:54:57,881 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family1/79acd90194484e429e64a28f2b3d5703 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/79acd90194484e429e64a28f2b3d5703 2023-03-24 21:54:57,886 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/0beec99d01ea409ab5eb29b007b9a14f, entries=2, sequenceid=17, filesize=5.1 K 2023-03-24 21:54:57,886 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/79acd90194484e429e64a28f2b3d5703, entries=4, sequenceid=17, filesize=5.4 K 2023-03-24 21:54:57,887 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 80c7f51f73f3bd5497dc15e88fadefa2 in 1326ms, sequenceid=17, compaction requested=true 2023-03-24 21:54:57,887 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:54:57,887 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:57,887 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:54:57,887 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family2/9f76b2a9f280438bb6a623955f71adae as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9f76b2a9f280438bb6a623955f71adae 2023-03-24 21:54:57,892 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9f76b2a9f280438bb6a623955f71adae, entries=2, sequenceid=17, filesize=5.1 K 2023-03-24 21:54:57,893 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for db6e510d903e066fe70066967a0093e2 in 1332ms, sequenceid=17, compaction requested=true 2023-03-24 21:54:57,893 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:54:57,893 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:57,893 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:54:57,893 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:54:57,893 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,893 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-24 21:54:57,893 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-24 21:54:57,895 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,895 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:57,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:57,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:57,896 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:57,896 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:57,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:57,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:57,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:57,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:57,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-24 21:54:57,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cec79fe[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:57,897 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-24 21:54:57,897 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:57,899 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:57,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:57,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:57,900 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-24 21:54:57,900 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,900 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:57,900 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-24 21:54:57,900 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:57,901 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-24 21:54:57,901 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:57,901 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,901 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-24 21:54:57,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:57,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:57,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:57,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:57,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:57,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:57,904 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-24 21:54:57,904 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:57,904 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:57,904 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-24 21:54:57,904 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-24 21:54:57,904 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:54:57,906 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:57,906 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:57,906 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:57,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:57,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:57,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:54:57,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:57,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:57,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:57,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:57,906 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,908 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:57,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:54:57,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,910 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:57,910 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:54:57,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:57,910 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:57,911 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:57,911 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-24 21:54:57,911 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:54:57,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-24 21:54:57,911 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:57,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:54:57,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:54:57,911 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:57,911 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:57,911 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:54:57,911 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:57,911 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:57,911 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:54:57,911 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:58,011 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:54:58,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:54:58,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1580): Client=jenkins//172.31.2.10 snapshot request for:{ ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-24 21:54:58,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1679694898027 (current time:1679694898027). 2023-03-24 21:54:58,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-24 21:54:58,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] zookeeper.ReadOnlyZKClient(139): Connect 0x5eb592cf to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:54:58,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2ac62411, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:54:58,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:54:58,045 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:43088, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:54:58,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5eb592cf to 127.0.0.1:54680 2023-03-24 21:54:58,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:54:58,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-24 21:54:58,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-24 21:54:58,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-24 21:54:58,059 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-24 21:54:58,060 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1679695498060 2023-03-24 21:54:58,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-24 21:54:58,062 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1679694898013 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-24 21:54:58,067 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-24 21:54:58,068 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-24 21:54:58,072 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-24 21:54:58,073 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-24 21:54:58,073 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 15 msec 2023-03-24 21:54:58,074 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-24 21:54:58,075 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1679695498075 2023-03-24 21:54:58,095 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1679694898013 2023-03-24 21:54:58,099 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1679694898013' 2023-03-24 21:54:58,099 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-24 21:54:58,099 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1679694898013' starting 'acquire' 2023-03-24 21:54:58,099 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1679694898013', kicking off acquire phase on members. 2023-03-24 21:54:58,100 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1679694898013 2023-03-24 21:54:58,100 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1679694898013 2023-03-24 21:54:58,107 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-24 21:54:58,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-24 21:54:58,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-24 21:54:58,107 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1679694898013/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1679694898013 2023-03-24 21:54:58,108 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1679694898013/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,108 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:58,108 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1679694898013 2023-03-24 21:54:58,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-03-24 21:54:58,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1679694898013 2023-03-24 21:54:58,109 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1679694898013 from table testCompactionWithHFileLink type FLUSH 2023-03-24 21:54:58,113 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1679694898013 2023-03-24 21:54:58,113 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1679694898013' with timeout 300000ms 2023-03-24 21:54:58,113 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-24 21:54:58,113 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1679694898013' starting 'acquire' stage 2023-03-24 21:54:58,113 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1679694898013' locally acquired 2023-03-24 21:54:58,113 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (snaptb-1679694898013) in zk 2023-03-24 21:54:58,117 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679694898013/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,117 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1679694898013 2023-03-24 21:54:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1679694898013/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-24 21:54:58,117 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1679694898013 2023-03-24 21:54:58,117 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1679694898013' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:54:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:58,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679694898013 2023-03-24 21:54:58,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:58,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'snaptb-1679694898013' on coordinator 2023-03-24 21:54:58,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a239e65[Count = 0] remaining members to acquire global barrier 2023-03-24 21:54:58,119 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1679694898013' starting 'in-barrier' execution. 2023-03-24 21:54:58,119 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1679694898013 2023-03-24 21:54:58,121 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679694898013 2023-03-24 21:54:58,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1679694898013 2023-03-24 21:54:58,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1679694898013 2023-03-24 21:54:58,121 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1679694898013' received 'reached' from coordinator. 2023-03-24 21:54:58,122 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1679694898013/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,122 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:54:58,123 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:58,123 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:58,123 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. started... 2023-03-24 21:54:58,123 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. started... 2023-03-24 21:54:58,123 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:54:58,123 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:54:58,125 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.' region-info for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,125 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.' region-info for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,127 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-03-24 21:54:58,127 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:58,127 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-03-24 21:54:58,127 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. started... 2023-03-24 21:54:58,127 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:54:58,127 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.' region-info for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,133 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-24 21:54:58,133 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-24 21:54:58,133 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-24 21:54:58,138 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/cd41d6bc7625452d8cd50d0509b4a453, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/57f179ca64e64316ba7cd9ca99b22557, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/00999c369b14425ea335da2d7d948d08, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/343affb19b444075983c87a317f71c54] hfiles 2023-03-24 21:54:58,138 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/cd41d6bc7625452d8cd50d0509b4a453 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,138 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/8dbaeb15a2174a16b5567bf58b9bbecd, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/0ade5f95824844f1bdb3bf6657afc66c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c2c3a5c11515473ba6a2ac8f75a53ddb, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/79acd90194484e429e64a28f2b3d5703] hfiles 2023-03-24 21:54:58,139 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/8dbaeb15a2174a16b5567bf58b9bbecd for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,138 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d27247c35aab4a5e95522f4b4082ee48, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/3a255a9bd6b34f9e82ea9b25a9f3431b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d9d7de1995344cbab674aca5eed6eee7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/502a23eefb564b108fd0804dd81ccc60] hfiles 2023-03-24 21:54:58,140 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d27247c35aab4a5e95522f4b4082ee48 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/0ade5f95824844f1bdb3bf6657afc66c for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/57f179ca64e64316ba7cd9ca99b22557 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/3a255a9bd6b34f9e82ea9b25a9f3431b for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/00999c369b14425ea335da2d7d948d08 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c2c3a5c11515473ba6a2ac8f75a53ddb for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/343affb19b444075983c87a317f71c54 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d9d7de1995344cbab674aca5eed6eee7 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/79acd90194484e429e64a28f2b3d5703 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/502a23eefb564b108fd0804dd81ccc60 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9430792f9a7c45b2b628cf070eabbb41, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/0eb237e6e4534356b0c3eb93951ca94b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/6b11f68343084dc3bc8a631c212cf15c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9f76b2a9f280438bb6a623955f71adae] hfiles 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/b0f3a1484555406892bf1e9f9963292d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/7e5ea937345a494e87b1c26c366f8cd7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/a87ee0a5f4f1401a967dae27a4650f1a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/0beec99d01ea409ab5eb29b007b9a14f] hfiles 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/265c8226a3a1414583f189917e8a53c7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/aad477eb45dc4b8fbb0b9c5cc5ab0982, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/ff2402da2b7244269d37afebd5825ed5, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/73842ebef8cc4e218b9f1e78ba7eead2] hfiles 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/265c8226a3a1414583f189917e8a53c7 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/b0f3a1484555406892bf1e9f9963292d for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9430792f9a7c45b2b628cf070eabbb41 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/7e5ea937345a494e87b1c26c366f8cd7 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/0eb237e6e4534356b0c3eb93951ca94b for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,142 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/aad477eb45dc4b8fbb0b9c5cc5ab0982 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,143 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/6b11f68343084dc3bc8a631c212cf15c for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,143 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/a87ee0a5f4f1401a967dae27a4650f1a for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,143 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9f76b2a9f280438bb6a623955f71adae for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,143 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/ff2402da2b7244269d37afebd5825ed5 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,143 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/0beec99d01ea409ab5eb29b007b9a14f for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,143 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/73842ebef8cc4e218b9f1e78ba7eead2 for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,168 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-24 21:54:58,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-24 21:54:58,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-24 21:54:58,172 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-24 21:54:58,372 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-24 21:54:58,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-24 21:54:58,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-24 21:54:58,374 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-24 21:54:58,585 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. completed. 2023-03-24 21:54:58,585 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:58,586 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-03-24 21:54:58,589 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. completed. 2023-03-24 21:54:58,589 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:58,589 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-03-24 21:54:58,605 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. completed. 2023-03-24 21:54:58,605 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:58,605 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-03-24 21:54:58,605 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-03-24 21:54:58,605 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,606 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1679694898013' locally completed 2023-03-24 21:54:58,606 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1679694898013' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:54:58,607 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679694898013/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,607 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1679694898013' has notified controller of completion 2023-03-24 21:54:58,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1679694898013/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:58,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-24 21:54:58,607 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:58,607 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1679694898013' completed. 2023-03-24 21:54:58,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679694898013 2023-03-24 21:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:58,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679694898013 2023-03-24 21:54:58,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1679694898013' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:54:58,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'snaptb-1679694898013', counting down latch. Waiting for 0 more 2023-03-24 21:54:58,611 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1679694898013' execution completed 2023-03-24 21:54:58,611 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:54:58,611 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:54:58,611 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1679694898013 2023-03-24 21:54:58,611 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1679694898013including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-03-24 21:54:58,613 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1679694898013 2023-03-24 21:54:58,613 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-24 21:54:58,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1679694898013 2023-03-24 21:54:58,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1679694898013 2023-03-24 21:54:58,613 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1679694898013 2023-03-24 21:54:58,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1679694898013 2023-03-24 21:54:58,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:54:58,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-24 21:54:58,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-24 21:54:58,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-24 21:54:58,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1679694898013 2023-03-24 21:54:58,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:54:58,615 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1679694898013/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679694898013 2023-03-24 21:54:58,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:54:58,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679694898013 2023-03-24 21:54:58,625 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1679694898013/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:54:58,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679694898013 2023-03-24 21:54:58,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,628 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679694898013/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,628 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-24 21:54:58,628 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679694898013 2023-03-24 21:54:58,628 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-24 21:54:58,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-24 21:54:58,628 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679694898013 2023-03-24 21:54:58,629 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:54:58,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-24 21:54:58,629 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679694898013/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:58,629 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1679694898013 2023-03-24 21:54:58,629 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679694898013 2023-03-24 21:54:58,629 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-03-24 21:54:58,629 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679694898013 2023-03-24 21:54:58,629 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1679694898013 2023-03-24 21:54:58,629 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1679694898013 2023-03-24 21:54:58,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1679694898013 because node does not exist (not an error) 2023-03-24 21:54:58,629 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-03-24 21:54:58,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-24 21:54:58,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-24 21:54:58,631 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303242e179998f2e24897b7653543ea105766, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303249102367c04aa4f6681d55ff092535183, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324d7da3a5c947a4742b6e70403b0b8915b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324d7fcdeeeaa884fb399b7825fb7a06335, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032408ac34e9f9cf436e9e176b1f391cb5cf, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303240938f5625b8f4728ae55dff1564edffb, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324ab93dd4f06c644be89323eed2735e5f0, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324dfed51c8c5474a239a39070aba3b8b49, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032438af3f7a90284a219128aa718c8b109e, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324af079d2fd1914f4fafbcb41bd9adc5b1, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324d0c5015a0307438da385076af4d9d54f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324ea6e832756384abbbdebb994807e4349] mob files 2023-03-24 21:54:58,631 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303242e179998f2e24897b7653543ea105766 2023-03-24 21:54:58,631 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303249102367c04aa4f6681d55ff092535183 2023-03-24 21:54:58,631 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324d7da3a5c947a4742b6e70403b0b8915b 2023-03-24 21:54:58,631 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324d7fcdeeeaa884fb399b7825fb7a06335 2023-03-24 21:54:58,631 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032408ac34e9f9cf436e9e176b1f391cb5cf 2023-03-24 21:54:58,631 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303240938f5625b8f4728ae55dff1564edffb 2023-03-24 21:54:58,631 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324ab93dd4f06c644be89323eed2735e5f0 2023-03-24 21:54:58,631 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324dfed51c8c5474a239a39070aba3b8b49 2023-03-24 21:54:58,631 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032438af3f7a90284a219128aa718c8b109e 2023-03-24 21:54:58,632 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324af079d2fd1914f4fafbcb41bd9adc5b1 2023-03-24 21:54:58,632 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324d0c5015a0307438da385076af4d9d54f 2023-03-24 21:54:58,632 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324ea6e832756384abbbdebb994807e4349 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303240a18c957ac64464aa452d87984ecd741, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303248f0be98635a74e8cb762dd3ac9d9dad1, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230324ae6b4dfca433486082c9e1f7e3e66627, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230324ee47c7143e3548a0946b7998233c5cb3, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032435e7af8aff134bc7978ad6fb7cd6c2be, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032436c0ae60afb848de99a9ba12d53edbbe, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324771fcbddd6164a379354a7c114971988, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324ee6b06f3c7fd4a228457033777494a55, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032434f646a8dbf04661a5b61817f4fadd69, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032448d400f1b8b64d3b8f83bec372eef9b3, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303245040c195a5574df4a60c60e92e90ed3a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230324e32db202e5394db1980382e90a3489c3] mob files 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303240a18c957ac64464aa452d87984ecd741 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303248f0be98635a74e8cb762dd3ac9d9dad1 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230324ae6b4dfca433486082c9e1f7e3e66627 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230324ee47c7143e3548a0946b7998233c5cb3 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032435e7af8aff134bc7978ad6fb7cd6c2be 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032436c0ae60afb848de99a9ba12d53edbbe 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324771fcbddd6164a379354a7c114971988 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324ee6b06f3c7fd4a228457033777494a55 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032434f646a8dbf04661a5b61817f4fadd69 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032448d400f1b8b64d3b8f83bec372eef9b3 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303245040c195a5574df4a60c60e92e90ed3a 2023-03-24 21:54:58,633 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230324e32db202e5394db1980382e90a3489c3 2023-03-24 21:54:58,644 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1679694898013 2023-03-24 21:54:58,647 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.hbase-snapshot/.tmp/snaptb-1679694898013 2023-03-24 21:54:58,674 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-24 21:54:58,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-24 21:54:58,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-24 21:54:58,675 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-24 21:54:58,734 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.hbase-snapshot/.tmp/snaptb-1679694898013 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.hbase-snapshot/snaptb-1679694898013 2023-03-24 21:54:58,765 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1679694898013 of table testCompactionWithHFileLink completed 2023-03-24 21:54:58,765 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.hbase-snapshot/.tmp/snaptb-1679694898013 2023-03-24 21:54:58,765 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.hbase-snapshot/.tmp/snaptb-1679694898013 2023-03-24 21:54:58,766 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1679694898013 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1679694898062Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1679694898644 (+582 ms)Consolidate snapshot: snaptb-1679694898013 at 1679694898644Loading Region manifests for snaptb-1679694898013 at 1679694898645 (+1 ms)Writing data manifest for snaptb-1679694898013 at 1679694898664 (+19 ms)Verifying snapshot: snaptb-1679694898013 at 1679694898692 (+28 ms)Snapshot snaptb-1679694898013 of table testCompactionWithHFileLink completed at 1679694898765 (+73 ms) 2023-03-24 21:54:58,767 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-24 21:54:58,768 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 696 msec 2023-03-24 21:54:59,176 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-24 21:54:59,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-24 21:54:59,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-24 21:54:59,210 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:54:59,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-24 21:54:59,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-24 21:54:59,212 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-24 21:54:59,212 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:59,212 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-24 21:54:59,212 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-24 21:54:59,213 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:59,213 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:59,215 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:54:59,215 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:59,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:54:59,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:54:59,215 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:54:59,215 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:54:59,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:59,215 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:54:59,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:54:59,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:54:59,216 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-24 21:54:59,216 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-24 21:54:59,216 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-24 21:54:59,216 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-24 21:54:59,216 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:54:59,216 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-24 21:54:59,216 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:54:59,216 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:54:59,216 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. started... 2023-03-24 21:54:59,216 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. started... 2023-03-24 21:54:59,217 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:59,217 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dd7d45dee3d2e56a7d8a4ae56820f04d 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-24 21:54:59,216 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:54:59,217 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:54:59,217 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. started... 2023-03-24 21:54:59,217 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing db6e510d903e066fe70066967a0093e2 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-24 21:54:59,217 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 80c7f51f73f3bd5497dc15e88fadefa2 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-24 21:54:59,293 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family1/8c85144fcf274b05b95bb81dfb2cd7a8 2023-03-24 21:54:59,301 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c85144fcf274b05b95bb81dfb2cd7a8 2023-03-24 21:54:59,689 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family1/fc8ab98a184b4fa781933d5f1ddcac56 2023-03-24 21:54:59,693 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc8ab98a184b4fa781933d5f1ddcac56 2023-03-24 21:54:59,738 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family2/354e4448fdaf41429d8bfc3f2a4114f6 2023-03-24 21:54:59,743 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 354e4448fdaf41429d8bfc3f2a4114f6 2023-03-24 21:54:59,744 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family1/8c85144fcf274b05b95bb81dfb2cd7a8 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/8c85144fcf274b05b95bb81dfb2cd7a8 2023-03-24 21:54:59,750 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c85144fcf274b05b95bb81dfb2cd7a8 2023-03-24 21:54:59,750 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/8c85144fcf274b05b95bb81dfb2cd7a8, entries=3, sequenceid=24, filesize=5.0 K 2023-03-24 21:54:59,751 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family2/354e4448fdaf41429d8bfc3f2a4114f6 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/354e4448fdaf41429d8bfc3f2a4114f6 2023-03-24 21:54:59,756 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 354e4448fdaf41429d8bfc3f2a4114f6 2023-03-24 21:54:59,756 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/354e4448fdaf41429d8bfc3f2a4114f6, entries=1, sequenceid=24, filesize=5.0 K 2023-03-24 21:54:59,757 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 80c7f51f73f3bd5497dc15e88fadefa2 in 540ms, sequenceid=24, compaction requested=true 2023-03-24 21:54:59,757 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:54:59,757 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:54:59,757 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:55:00,087 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family1/8996c6bf02054682bec5c4aad134295c 2023-03-24 21:55:00,091 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8996c6bf02054682bec5c4aad134295c 2023-03-24 21:55:00,108 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family2/584ee7367b7f45ab90c3fb63673453c8 2023-03-24 21:55:00,112 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 584ee7367b7f45ab90c3fb63673453c8 2023-03-24 21:55:00,113 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family1/fc8ab98a184b4fa781933d5f1ddcac56 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/fc8ab98a184b4fa781933d5f1ddcac56 2023-03-24 21:55:00,117 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc8ab98a184b4fa781933d5f1ddcac56 2023-03-24 21:55:00,117 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/fc8ab98a184b4fa781933d5f1ddcac56, entries=3, sequenceid=24, filesize=5.0 K 2023-03-24 21:55:00,118 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family2/584ee7367b7f45ab90c3fb63673453c8 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/584ee7367b7f45ab90c3fb63673453c8 2023-03-24 21:55:00,124 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 584ee7367b7f45ab90c3fb63673453c8 2023-03-24 21:55:00,124 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/584ee7367b7f45ab90c3fb63673453c8, entries=1, sequenceid=24, filesize=5.0 K 2023-03-24 21:55:00,125 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for db6e510d903e066fe70066967a0093e2 in 908ms, sequenceid=24, compaction requested=true 2023-03-24 21:55:00,126 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:55:00,126 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:00,126 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:55:00,506 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family2/40276b91b21a48ca9e1ef4a14f0429dd 2023-03-24 21:55:00,512 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 40276b91b21a48ca9e1ef4a14f0429dd 2023-03-24 21:55:00,513 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family1/8996c6bf02054682bec5c4aad134295c as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/8996c6bf02054682bec5c4aad134295c 2023-03-24 21:55:00,518 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8996c6bf02054682bec5c4aad134295c 2023-03-24 21:55:00,519 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/8996c6bf02054682bec5c4aad134295c, entries=3, sequenceid=24, filesize=5.0 K 2023-03-24 21:55:00,519 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family2/40276b91b21a48ca9e1ef4a14f0429dd as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/40276b91b21a48ca9e1ef4a14f0429dd 2023-03-24 21:55:00,524 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 40276b91b21a48ca9e1ef4a14f0429dd 2023-03-24 21:55:00,524 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/40276b91b21a48ca9e1ef4a14f0429dd, entries=1, sequenceid=24, filesize=5.0 K 2023-03-24 21:55:00,525 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for dd7d45dee3d2e56a7d8a4ae56820f04d in 1308ms, sequenceid=24, compaction requested=true 2023-03-24 21:55:00,526 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:55:00,526 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:00,526 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:55:00,526 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:55:00,526 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,526 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-24 21:55:00,526 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-24 21:55:00,528 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,528 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:00,528 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:55:00,528 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:55:00,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:00,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-24 21:55:00,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b56e33b[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:00,530 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-24 21:55:00,530 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:55:00,533 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:55:00,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:55:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:55:00,533 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-24 21:55:00,533 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-24 21:55:00,533 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:00,533 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,533 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:00,537 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-24 21:55:00,538 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:00,537 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,538 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-24 21:55:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:55:00,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:00,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:55:00,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:00,540 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-24 21:55:00,540 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-24 21:55:00,540 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:00,540 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:00,540 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-24 21:55:00,540 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:00,542 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:55:00,542 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:55:00,542 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:00,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:55:00,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:55:00,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:55:00,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:00,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:00,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:00,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:00,543 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:00,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:55:00,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:55:00,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:00,544 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:55:00,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:00,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-24 21:55:00,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,547 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:00,547 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:00,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:00,547 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:00,547 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:55:00,547 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:00,547 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-24 21:55:00,548 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:00,548 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:55:00,548 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-24 21:55:00,548 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-24 21:55:00,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-24 21:55:00,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:00,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:00,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:00,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-24 21:55:00,548 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:00,548 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:00,648 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:00,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:00,650 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:00,650 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): dd7d45dee3d2e56a7d8a4ae56820f04d/family1 is initiating major compaction (all files) 2023-03-24 21:55:00,650 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:00,651 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:00,651 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of dd7d45dee3d2e56a7d8a4ae56820f04d/family1 in testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:00,651 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d27247c35aab4a5e95522f4b4082ee48, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/3a255a9bd6b34f9e82ea9b25a9f3431b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d9d7de1995344cbab674aca5eed6eee7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/502a23eefb564b108fd0804dd81ccc60, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/8996c6bf02054682bec5c4aad134295c] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp, totalSize=26.7 K 2023-03-24 21:55:00,651 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting d27247c35aab4a5e95522f4b4082ee48, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694892685 2023-03-24 21:55:00,652 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 3a255a9bd6b34f9e82ea9b25a9f3431b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694893924 2023-03-24 21:55:00,652 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting d9d7de1995344cbab674aca5eed6eee7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694894626 2023-03-24 21:55:00,653 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 502a23eefb564b108fd0804dd81ccc60, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694896542 2023-03-24 21:55:00,653 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 8996c6bf02054682bec5c4aad134295c, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-24 21:55:00,680 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): dd7d45dee3d2e56a7d8a4ae56820f04d#family1#compaction#107 average throughput is 0.24 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:00,696 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303247ef9bb31ee904076b6636a5e33113552_del 2023-03-24 21:55:00,696 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303247ef9bb31ee904076b6636a5e33113552_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303247ef9bb31ee904076b6636a5e33113552_del 2023-03-24 21:55:01,116 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4231f470a5304731954c12ad8206e2b1 2023-03-24 21:55:01,117 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family1/4231f470a5304731954c12ad8206e2b1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/4231f470a5304731954c12ad8206e2b1 2023-03-24 21:55:01,121 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4231f470a5304731954c12ad8206e2b1 2023-03-24 21:55:01,122 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in dd7d45dee3d2e56a7d8a4ae56820f04d/family1 of dd7d45dee3d2e56a7d8a4ae56820f04d into 4231f470a5304731954c12ad8206e2b1(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:01,123 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:55:01,123 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:01,123 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): dd7d45dee3d2e56a7d8a4ae56820f04d/family2 is initiating major compaction (all files) 2023-03-24 21:55:01,123 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:01,123 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:01,124 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of dd7d45dee3d2e56a7d8a4ae56820f04d/family2 in testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:01,124 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/265c8226a3a1414583f189917e8a53c7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/aad477eb45dc4b8fbb0b9c5cc5ab0982, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/ff2402da2b7244269d37afebd5825ed5, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/73842ebef8cc4e218b9f1e78ba7eead2, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/40276b91b21a48ca9e1ef4a14f0429dd] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp, totalSize=25.4 K 2023-03-24 21:55:01,125 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 265c8226a3a1414583f189917e8a53c7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694892685 2023-03-24 21:55:01,125 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting aad477eb45dc4b8fbb0b9c5cc5ab0982, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694893924 2023-03-24 21:55:01,126 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting ff2402da2b7244269d37afebd5825ed5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694894626 2023-03-24 21:55:01,126 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 73842ebef8cc4e218b9f1e78ba7eead2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694896542 2023-03-24 21:55:01,126 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 40276b91b21a48ca9e1ef4a14f0429dd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-24 21:55:01,139 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): dd7d45dee3d2e56a7d8a4ae56820f04d#family2#compaction#108 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:01,157 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303244f7053aa59ef4aabacfbdad9790d4b25_del 2023-03-24 21:55:01,157 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303244f7053aa59ef4aabacfbdad9790d4b25_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303244f7053aa59ef4aabacfbdad9790d4b25_del 2023-03-24 21:55:01,164 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d27247c35aab4a5e95522f4b4082ee48, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/3a255a9bd6b34f9e82ea9b25a9f3431b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d9d7de1995344cbab674aca5eed6eee7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/502a23eefb564b108fd0804dd81ccc60, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/8996c6bf02054682bec5c4aad134295c] to archive 2023-03-24 21:55:01,164 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:01,176 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d27247c35aab4a5e95522f4b4082ee48 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d27247c35aab4a5e95522f4b4082ee48 2023-03-24 21:55:01,178 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/3a255a9bd6b34f9e82ea9b25a9f3431b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/3a255a9bd6b34f9e82ea9b25a9f3431b 2023-03-24 21:55:01,179 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d9d7de1995344cbab674aca5eed6eee7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/d9d7de1995344cbab674aca5eed6eee7 2023-03-24 21:55:01,181 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/502a23eefb564b108fd0804dd81ccc60 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/502a23eefb564b108fd0804dd81ccc60 2023-03-24 21:55:01,182 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/8996c6bf02054682bec5c4aad134295c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/8996c6bf02054682bec5c4aad134295c 2023-03-24 21:55:01,593 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 240fee55058242128795398a51ecd239 2023-03-24 21:55:01,594 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family2/240fee55058242128795398a51ecd239 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/240fee55058242128795398a51ecd239 2023-03-24 21:55:01,599 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 240fee55058242128795398a51ecd239 2023-03-24 21:55:01,600 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in dd7d45dee3d2e56a7d8a4ae56820f04d/family2 of dd7d45dee3d2e56a7d8a4ae56820f04d into 240fee55058242128795398a51ecd239(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:01,601 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:55:01,601 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:01,601 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): db6e510d903e066fe70066967a0093e2/family1 is initiating major compaction (all files) 2023-03-24 21:55:01,601 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:01,601 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:01,602 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of db6e510d903e066fe70066967a0093e2/family1 in testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:01,602 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/8dbaeb15a2174a16b5567bf58b9bbecd, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/0ade5f95824844f1bdb3bf6657afc66c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c2c3a5c11515473ba6a2ac8f75a53ddb, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/79acd90194484e429e64a28f2b3d5703, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/fc8ab98a184b4fa781933d5f1ddcac56] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp, totalSize=26.7 K 2023-03-24 21:55:01,603 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 8dbaeb15a2174a16b5567bf58b9bbecd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694892686 2023-03-24 21:55:01,604 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 0ade5f95824844f1bdb3bf6657afc66c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694893924 2023-03-24 21:55:01,605 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting c2c3a5c11515473ba6a2ac8f75a53ddb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694894626 2023-03-24 21:55:01,605 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 79acd90194484e429e64a28f2b3d5703, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694896542 2023-03-24 21:55:01,606 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting fc8ab98a184b4fa781933d5f1ddcac56, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-24 21:55:01,618 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): db6e510d903e066fe70066967a0093e2#family1#compaction#109 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:01,662 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/265c8226a3a1414583f189917e8a53c7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/aad477eb45dc4b8fbb0b9c5cc5ab0982, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/ff2402da2b7244269d37afebd5825ed5, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/73842ebef8cc4e218b9f1e78ba7eead2, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/40276b91b21a48ca9e1ef4a14f0429dd] to archive 2023-03-24 21:55:01,663 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:01,665 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/265c8226a3a1414583f189917e8a53c7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/265c8226a3a1414583f189917e8a53c7 2023-03-24 21:55:01,666 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/aad477eb45dc4b8fbb0b9c5cc5ab0982 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/aad477eb45dc4b8fbb0b9c5cc5ab0982 2023-03-24 21:55:01,667 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/ff2402da2b7244269d37afebd5825ed5 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/ff2402da2b7244269d37afebd5825ed5 2023-03-24 21:55:01,669 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/73842ebef8cc4e218b9f1e78ba7eead2 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/73842ebef8cc4e218b9f1e78ba7eead2 2023-03-24 21:55:01,670 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/40276b91b21a48ca9e1ef4a14f0429dd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/40276b91b21a48ca9e1ef4a14f0429dd 2023-03-24 21:55:02,046 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230324a1aa9ac263af4200baa907644a89abb8_del 2023-03-24 21:55:02,047 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324a1aa9ac263af4200baa907644a89abb8_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324a1aa9ac263af4200baa907644a89abb8_del 2023-03-24 21:55:02,463 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c064c18447d14ed7a695f7968924001a 2023-03-24 21:55:02,464 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family1/c064c18447d14ed7a695f7968924001a as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c064c18447d14ed7a695f7968924001a 2023-03-24 21:55:02,469 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c064c18447d14ed7a695f7968924001a 2023-03-24 21:55:02,470 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in db6e510d903e066fe70066967a0093e2/family1 of db6e510d903e066fe70066967a0093e2 into c064c18447d14ed7a695f7968924001a(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:02,471 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:55:02,471 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:02,471 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): db6e510d903e066fe70066967a0093e2/family2 is initiating major compaction (all files) 2023-03-24 21:55:02,471 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:02,471 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:02,472 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of db6e510d903e066fe70066967a0093e2/family2 in testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:02,472 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9430792f9a7c45b2b628cf070eabbb41, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/0eb237e6e4534356b0c3eb93951ca94b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/6b11f68343084dc3bc8a631c212cf15c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9f76b2a9f280438bb6a623955f71adae, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/584ee7367b7f45ab90c3fb63673453c8] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp, totalSize=25.4 K 2023-03-24 21:55:02,473 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 9430792f9a7c45b2b628cf070eabbb41, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694892686 2023-03-24 21:55:02,473 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 0eb237e6e4534356b0c3eb93951ca94b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694893924 2023-03-24 21:55:02,473 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 6b11f68343084dc3bc8a631c212cf15c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694894626 2023-03-24 21:55:02,474 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 9f76b2a9f280438bb6a623955f71adae, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694896542 2023-03-24 21:55:02,475 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 584ee7367b7f45ab90c3fb63673453c8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-24 21:55:02,490 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): db6e510d903e066fe70066967a0093e2#family2#compaction#110 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:02,566 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/8dbaeb15a2174a16b5567bf58b9bbecd, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/0ade5f95824844f1bdb3bf6657afc66c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c2c3a5c11515473ba6a2ac8f75a53ddb, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/79acd90194484e429e64a28f2b3d5703, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/fc8ab98a184b4fa781933d5f1ddcac56] to archive 2023-03-24 21:55:02,567 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:02,569 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/8dbaeb15a2174a16b5567bf58b9bbecd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/8dbaeb15a2174a16b5567bf58b9bbecd 2023-03-24 21:55:02,572 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/0ade5f95824844f1bdb3bf6657afc66c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/0ade5f95824844f1bdb3bf6657afc66c 2023-03-24 21:55:02,573 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c2c3a5c11515473ba6a2ac8f75a53ddb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c2c3a5c11515473ba6a2ac8f75a53ddb 2023-03-24 21:55:02,574 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/79acd90194484e429e64a28f2b3d5703 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/79acd90194484e429e64a28f2b3d5703 2023-03-24 21:55:02,575 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/fc8ab98a184b4fa781933d5f1ddcac56 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/fc8ab98a184b4fa781933d5f1ddcac56 2023-03-24 21:55:02,908 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230324a75914ff0ba04d5ca4adc2fe4eaab8d7_del 2023-03-24 21:55:02,909 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324a75914ff0ba04d5ca4adc2fe4eaab8d7_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324a75914ff0ba04d5ca4adc2fe4eaab8d7_del 2023-03-24 21:55:03,336 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3a0abc512f3c40e4bb0b098a5d5a6367 2023-03-24 21:55:03,337 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family2/3a0abc512f3c40e4bb0b098a5d5a6367 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/3a0abc512f3c40e4bb0b098a5d5a6367 2023-03-24 21:55:03,342 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3a0abc512f3c40e4bb0b098a5d5a6367 2023-03-24 21:55:03,344 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in db6e510d903e066fe70066967a0093e2/family2 of db6e510d903e066fe70066967a0093e2 into 3a0abc512f3c40e4bb0b098a5d5a6367(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:03,344 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:55:03,344 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:03,344 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): 80c7f51f73f3bd5497dc15e88fadefa2/family1 is initiating major compaction (all files) 2023-03-24 21:55:03,344 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:03,345 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:03,346 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of 80c7f51f73f3bd5497dc15e88fadefa2/family1 in testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:03,346 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/cd41d6bc7625452d8cd50d0509b4a453, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/57f179ca64e64316ba7cd9ca99b22557, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/00999c369b14425ea335da2d7d948d08, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/343affb19b444075983c87a317f71c54, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/8c85144fcf274b05b95bb81dfb2cd7a8] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp, totalSize=26.7 K 2023-03-24 21:55:03,347 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting cd41d6bc7625452d8cd50d0509b4a453, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694892684 2023-03-24 21:55:03,347 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 57f179ca64e64316ba7cd9ca99b22557, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694893924 2023-03-24 21:55:03,348 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 00999c369b14425ea335da2d7d948d08, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694894625 2023-03-24 21:55:03,348 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 343affb19b444075983c87a317f71c54, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694896541 2023-03-24 21:55:03,349 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 8c85144fcf274b05b95bb81dfb2cd7a8, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-24 21:55:03,363 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9430792f9a7c45b2b628cf070eabbb41, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/0eb237e6e4534356b0c3eb93951ca94b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/6b11f68343084dc3bc8a631c212cf15c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9f76b2a9f280438bb6a623955f71adae, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/584ee7367b7f45ab90c3fb63673453c8] to archive 2023-03-24 21:55:03,364 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:03,366 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9430792f9a7c45b2b628cf070eabbb41 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9430792f9a7c45b2b628cf070eabbb41 2023-03-24 21:55:03,368 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/0eb237e6e4534356b0c3eb93951ca94b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/0eb237e6e4534356b0c3eb93951ca94b 2023-03-24 21:55:03,369 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/6b11f68343084dc3bc8a631c212cf15c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/6b11f68343084dc3bc8a631c212cf15c 2023-03-24 21:55:03,370 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9f76b2a9f280438bb6a623955f71adae to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/9f76b2a9f280438bb6a623955f71adae 2023-03-24 21:55:03,371 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/584ee7367b7f45ab90c3fb63673453c8 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/584ee7367b7f45ab90c3fb63673453c8 2023-03-24 21:55:03,381 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): 80c7f51f73f3bd5497dc15e88fadefa2#family1#compaction#111 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:03,394 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303240ca6ba86666b4d7986aa6358f29048c4_del 2023-03-24 21:55:03,395 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303240ca6ba86666b4d7986aa6358f29048c4_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303240ca6ba86666b4d7986aa6358f29048c4_del 2023-03-24 21:55:04,213 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d837959eb97042db907d47983a1ad1f9 2023-03-24 21:55:04,214 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family1/d837959eb97042db907d47983a1ad1f9 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/d837959eb97042db907d47983a1ad1f9 2023-03-24 21:55:04,219 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d837959eb97042db907d47983a1ad1f9 2023-03-24 21:55:04,222 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 80c7f51f73f3bd5497dc15e88fadefa2/family1 of 80c7f51f73f3bd5497dc15e88fadefa2 into d837959eb97042db907d47983a1ad1f9(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:04,222 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:55:04,223 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:04,223 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): 80c7f51f73f3bd5497dc15e88fadefa2/family2 is initiating major compaction (all files) 2023-03-24 21:55:04,223 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:04,223 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:04,224 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of 80c7f51f73f3bd5497dc15e88fadefa2/family2 in testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:04,224 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/b0f3a1484555406892bf1e9f9963292d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/7e5ea937345a494e87b1c26c366f8cd7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/a87ee0a5f4f1401a967dae27a4650f1a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/0beec99d01ea409ab5eb29b007b9a14f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/354e4448fdaf41429d8bfc3f2a4114f6] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp, totalSize=25.4 K 2023-03-24 21:55:04,225 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting b0f3a1484555406892bf1e9f9963292d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694892684 2023-03-24 21:55:04,225 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 7e5ea937345a494e87b1c26c366f8cd7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694893924 2023-03-24 21:55:04,226 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting a87ee0a5f4f1401a967dae27a4650f1a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694894625 2023-03-24 21:55:04,226 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 0beec99d01ea409ab5eb29b007b9a14f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694896541 2023-03-24 21:55:04,227 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 354e4448fdaf41429d8bfc3f2a4114f6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-24 21:55:04,259 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): 80c7f51f73f3bd5497dc15e88fadefa2#family2#compaction#112 average throughput is 0.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:04,264 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/cd41d6bc7625452d8cd50d0509b4a453, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/57f179ca64e64316ba7cd9ca99b22557, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/00999c369b14425ea335da2d7d948d08, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/343affb19b444075983c87a317f71c54, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/8c85144fcf274b05b95bb81dfb2cd7a8] to archive 2023-03-24 21:55:04,265 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:04,272 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/cd41d6bc7625452d8cd50d0509b4a453 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/cd41d6bc7625452d8cd50d0509b4a453 2023-03-24 21:55:04,274 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/57f179ca64e64316ba7cd9ca99b22557 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/57f179ca64e64316ba7cd9ca99b22557 2023-03-24 21:55:04,276 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/00999c369b14425ea335da2d7d948d08 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/00999c369b14425ea335da2d7d948d08 2023-03-24 21:55:04,276 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303243822b4be036143d79f8833a8e23ac8ec_del 2023-03-24 21:55:04,276 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303243822b4be036143d79f8833a8e23ac8ec_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303243822b4be036143d79f8833a8e23ac8ec_del 2023-03-24 21:55:04,278 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/343affb19b444075983c87a317f71c54 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/343affb19b444075983c87a317f71c54 2023-03-24 21:55:04,280 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/8c85144fcf274b05b95bb81dfb2cd7a8 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/8c85144fcf274b05b95bb81dfb2cd7a8 2023-03-24 21:55:04,698 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca69b78c3ca24abd91a01e821b6d1cec 2023-03-24 21:55:04,707 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family2/ca69b78c3ca24abd91a01e821b6d1cec as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/ca69b78c3ca24abd91a01e821b6d1cec 2023-03-24 21:55:04,713 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca69b78c3ca24abd91a01e821b6d1cec 2023-03-24 21:55:04,714 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 80c7f51f73f3bd5497dc15e88fadefa2/family2 of 80c7f51f73f3bd5497dc15e88fadefa2 into ca69b78c3ca24abd91a01e821b6d1cec(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:04,714 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:55:04,720 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:04,724 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:04,729 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:04,730 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:04,736 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-24 21:55:04,762 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/b0f3a1484555406892bf1e9f9963292d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/7e5ea937345a494e87b1c26c366f8cd7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/a87ee0a5f4f1401a967dae27a4650f1a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/0beec99d01ea409ab5eb29b007b9a14f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/354e4448fdaf41429d8bfc3f2a4114f6] to archive 2023-03-24 21:55:04,763 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:04,764 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/b0f3a1484555406892bf1e9f9963292d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/b0f3a1484555406892bf1e9f9963292d 2023-03-24 21:55:04,765 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/7e5ea937345a494e87b1c26c366f8cd7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/7e5ea937345a494e87b1c26c366f8cd7 2023-03-24 21:55:04,767 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/a87ee0a5f4f1401a967dae27a4650f1a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/a87ee0a5f4f1401a967dae27a4650f1a 2023-03-24 21:55:04,768 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/0beec99d01ea409ab5eb29b007b9a14f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/0beec99d01ea409ab5eb29b007b9a14f 2023-03-24 21:55:04,768 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-24 21:55:04,769 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/354e4448fdaf41429d8bfc3f2a4114f6 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/354e4448fdaf41429d8bfc3f2a4114f6 2023-03-24 21:55:04,771 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303240ca6ba86666b4d7986aa6358f29048c4_del 2023-03-24 21:55:04,775 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303247ef9bb31ee904076b6636a5e33113552_del 2023-03-24 21:55:04,778 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230324a1aa9ac263af4200baa907644a89abb8_del 2023-03-24 21:55:04,778 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:04,778 DEBUG [Listener at localhost/39907] zookeeper.ReadOnlyZKClient(139): Connect 0x28a98f9d to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:55:04,783 DEBUG [Listener at localhost/39907] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@371a83a4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:55:04,784 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230324 for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:04,784 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230324 for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:04,784 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230324 for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:04,839 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324780fde5425b04acdb90ab96751a03e42 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324780fde5425b04acdb90ab96751a03e42 2023-03-24 21:55:04,840 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230324 table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:04,853 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:55:04,856 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:39824, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:55:04,863 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:55:04,865 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:53192, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:55:04,873 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230324/family1/18f57d8c35744a8ab3ef190dbaf53c21 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-24 21:55:04,873 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230324/family1/18f57d8c35744a8ab3ef190dbaf53c21}] 2023-03-24 21:55:04,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230324/family1/18f57d8c35744a8ab3ef190dbaf53c21 for inclusion in db6e510d903e066fe70066967a0093e2/family1 2023-03-24 21:55:04,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-24 21:55:04,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-24 21:55:04,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:55:04,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230324/family1/18f57d8c35744a8ab3ef190dbaf53c21 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__lheuuvrgn3pc90k52tfmis4jpvmun75k41095lvl8f98ppglk0cm7drb4j0tk71t/family1/18f57d8c35744a8ab3ef190dbaf53c21 2023-03-24 21:55:04,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__lheuuvrgn3pc90k52tfmis4jpvmun75k41095lvl8f98ppglk0cm7drb4j0tk71t/family1/18f57d8c35744a8ab3ef190dbaf53c21 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/71d0651323034e5eac256c0f5f19f013_SeqId_29_ 2023-03-24 21:55:04,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__lheuuvrgn3pc90k52tfmis4jpvmun75k41095lvl8f98ppglk0cm7drb4j0tk71t/family1/18f57d8c35744a8ab3ef190dbaf53c21 into db6e510d903e066fe70066967a0093e2/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/71d0651323034e5eac256c0f5f19f013_SeqId_29_ - updating store file list. 2023-03-24 21:55:04,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/71d0651323034e5eac256c0f5f19f013_SeqId_29_ into db6e510d903e066fe70066967a0093e2/family1 2023-03-24 21:55:04,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__lheuuvrgn3pc90k52tfmis4jpvmun75k41095lvl8f98ppglk0cm7drb4j0tk71t/family1/18f57d8c35744a8ab3ef190dbaf53c21 into db6e510d903e066fe70066967a0093e2/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/71d0651323034e5eac256c0f5f19f013_SeqId_29_) 2023-03-24 21:55:04,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__lheuuvrgn3pc90k52tfmis4jpvmun75k41095lvl8f98ppglk0cm7drb4j0tk71t/family1/18f57d8c35744a8ab3ef190dbaf53c21 2023-03-24 21:55:04,887 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230324 table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:04,888 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:04,889 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032408ac34e9f9cf436e9e176b1f391cb5cf to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032408ac34e9f9cf436e9e176b1f391cb5cf 2023-03-24 21:55:04,889 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303240938f5625b8f4728ae55dff1564edffb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303240938f5625b8f4728ae55dff1564edffb 2023-03-24 21:55:04,890 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324ab93dd4f06c644be89323eed2735e5f0 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324ab93dd4f06c644be89323eed2735e5f0 2023-03-24 21:55:04,891 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324dfed51c8c5474a239a39070aba3b8b49 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324dfed51c8c5474a239a39070aba3b8b49 2023-03-24 21:55:04,891 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-24 21:55:04,891 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230324' 2023-03-24 21:55:04,891 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230324 for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:05,635 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324c43eee7499b548a08101026b2071f356 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324c43eee7499b548a08101026b2071f356 2023-03-24 21:55:05,636 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230324 table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:05,644 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324ca2c9bc1a3594a67a55d343ad56c7a79 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324ca2c9bc1a3594a67a55d343ad56c7a79 2023-03-24 21:55:05,645 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230324 table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:05,661 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230324/family1/839b1f3591964fd98577d95da753f8d1 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-24 21:55:05,663 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230324/family1/839b1f3591964fd98577d95da753f8d1}] 2023-03-24 21:55:05,666 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230324/family1/839b1f3591964fd98577d95da753f8d1 for inclusion in dd7d45dee3d2e56a7d8a4ae56820f04d/family1 2023-03-24 21:55:05,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-24 21:55:05,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-24 21:55:05,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:55:05,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230324/family1/839b1f3591964fd98577d95da753f8d1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__ejisng0dmessaon9eetaa6kv6mdadsfb9qefc8nb7d6ri02luf4cnfp93qip0ead/family1/839b1f3591964fd98577d95da753f8d1 2023-03-24 21:55:05,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__ejisng0dmessaon9eetaa6kv6mdadsfb9qefc8nb7d6ri02luf4cnfp93qip0ead/family1/839b1f3591964fd98577d95da753f8d1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/9179e35c711e43a4ac15fd4ac2a8f01b_SeqId_29_ 2023-03-24 21:55:05,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__ejisng0dmessaon9eetaa6kv6mdadsfb9qefc8nb7d6ri02luf4cnfp93qip0ead/family1/839b1f3591964fd98577d95da753f8d1 into dd7d45dee3d2e56a7d8a4ae56820f04d/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/9179e35c711e43a4ac15fd4ac2a8f01b_SeqId_29_ - updating store file list. 2023-03-24 21:55:05,676 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230324/family1/3b4fbc64b0914ac1bed9f481786cad84 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-24 21:55:05,677 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230324/family1/3b4fbc64b0914ac1bed9f481786cad84}] 2023-03-24 21:55:05,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/9179e35c711e43a4ac15fd4ac2a8f01b_SeqId_29_ into dd7d45dee3d2e56a7d8a4ae56820f04d/family1 2023-03-24 21:55:05,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__ejisng0dmessaon9eetaa6kv6mdadsfb9qefc8nb7d6ri02luf4cnfp93qip0ead/family1/839b1f3591964fd98577d95da753f8d1 into dd7d45dee3d2e56a7d8a4ae56820f04d/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/9179e35c711e43a4ac15fd4ac2a8f01b_SeqId_29_) 2023-03-24 21:55:05,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__ejisng0dmessaon9eetaa6kv6mdadsfb9qefc8nb7d6ri02luf4cnfp93qip0ead/family1/839b1f3591964fd98577d95da753f8d1 2023-03-24 21:55:05,679 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230324/family1/3b4fbc64b0914ac1bed9f481786cad84 for inclusion in 80c7f51f73f3bd5497dc15e88fadefa2/family1 2023-03-24 21:55:05,680 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230324 table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:05,680 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:05,681 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303242e179998f2e24897b7653543ea105766 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303242e179998f2e24897b7653543ea105766 2023-03-24 21:55:05,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-24 21:55:05,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last=1 2023-03-24 21:55:05,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:55:05,682 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303249102367c04aa4f6681d55ff092535183 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303249102367c04aa4f6681d55ff092535183 2023-03-24 21:55:05,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230324/family1/3b4fbc64b0914ac1bed9f481786cad84 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__vn4q22ro078a1agbv4lc7odmcc79nl7dnj390oo7465fednqvk63c2rhshpkpkug/family1/3b4fbc64b0914ac1bed9f481786cad84 2023-03-24 21:55:05,683 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324d7da3a5c947a4742b6e70403b0b8915b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324d7da3a5c947a4742b6e70403b0b8915b 2023-03-24 21:55:05,684 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324d7fcdeeeaa884fb399b7825fb7a06335 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324d7fcdeeeaa884fb399b7825fb7a06335 2023-03-24 21:55:05,684 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-24 21:55:05,684 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230324' 2023-03-24 21:55:05,684 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230324 for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:05,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__vn4q22ro078a1agbv4lc7odmcc79nl7dnj390oo7465fednqvk63c2rhshpkpkug/family1/3b4fbc64b0914ac1bed9f481786cad84 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/313a08f3040d4b1a82a70be8be3a3d51_SeqId_29_ 2023-03-24 21:55:05,685 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__vn4q22ro078a1agbv4lc7odmcc79nl7dnj390oo7465fednqvk63c2rhshpkpkug/family1/3b4fbc64b0914ac1bed9f481786cad84 into 80c7f51f73f3bd5497dc15e88fadefa2/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/313a08f3040d4b1a82a70be8be3a3d51_SeqId_29_ - updating store file list. 2023-03-24 21:55:05,687 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/313a08f3040d4b1a82a70be8be3a3d51_SeqId_29_ into 80c7f51f73f3bd5497dc15e88fadefa2/family1 2023-03-24 21:55:05,687 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__vn4q22ro078a1agbv4lc7odmcc79nl7dnj390oo7465fednqvk63c2rhshpkpkug/family1/3b4fbc64b0914ac1bed9f481786cad84 into 80c7f51f73f3bd5497dc15e88fadefa2/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/313a08f3040d4b1a82a70be8be3a3d51_SeqId_29_) 2023-03-24 21:55:05,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__vn4q22ro078a1agbv4lc7odmcc79nl7dnj390oo7465fednqvk63c2rhshpkpkug/family1/3b4fbc64b0914ac1bed9f481786cad84 2023-03-24 21:55:05,689 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230324 table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:05,690 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:05,690 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032438af3f7a90284a219128aa718c8b109e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032438af3f7a90284a219128aa718c8b109e 2023-03-24 21:55:05,691 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324af079d2fd1914f4fafbcb41bd9adc5b1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324af079d2fd1914f4fafbcb41bd9adc5b1 2023-03-24 21:55:05,692 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324d0c5015a0307438da385076af4d9d54f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324d0c5015a0307438da385076af4d9d54f 2023-03-24 21:55:05,692 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324ea6e832756384abbbdebb994807e4349 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324ea6e832756384abbbdebb994807e4349 2023-03-24 21:55:05,692 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-24 21:55:05,693 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230324' 2023-03-24 21:55:05,693 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230324 for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:05,693 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:05,693 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:05,693 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(367): [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303240ca6ba86666b4d7986aa6358f29048c4_del] 2023-03-24 21:55:05,693 DEBUG [Listener at localhost/39907] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:05,694 DEBUG [Listener at localhost/39907] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303240ca6ba86666b4d7986aa6358f29048c4_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303240ca6ba86666b4d7986aa6358f29048c4_del 2023-03-24 21:55:05,694 INFO [Listener at localhost/39907] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-24 21:55:05,695 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(367): [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303247ef9bb31ee904076b6636a5e33113552_del] 2023-03-24 21:55:05,695 DEBUG [Listener at localhost/39907] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:05,696 DEBUG [Listener at localhost/39907] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303247ef9bb31ee904076b6636a5e33113552_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303247ef9bb31ee904076b6636a5e33113552_del 2023-03-24 21:55:05,696 INFO [Listener at localhost/39907] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-24 21:55:05,696 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(367): [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324a1aa9ac263af4200baa907644a89abb8_del] 2023-03-24 21:55:05,696 DEBUG [Listener at localhost/39907] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:05,697 DEBUG [Listener at localhost/39907] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324a1aa9ac263af4200baa907644a89abb8_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324a1aa9ac263af4200baa907644a89abb8_del 2023-03-24 21:55:05,697 INFO [Listener at localhost/39907] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-24 21:55:05,699 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:05,702 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:05,706 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:05,707 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:05,712 INFO [Listener at localhost/39907] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-03-24 21:55:05,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$11(2418): Client=jenkins//172.31.2.10 disable testCompactionWithHFileLink 2023-03-24 21:55:05,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-03-24 21:55:05,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-24 21:55:05,715 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694905715"}]},"ts":"1679694905715"} 2023-03-24 21:55:05,716 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-03-24 21:55:05,717 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-03-24 21:55:05,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=80c7f51f73f3bd5497dc15e88fadefa2, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd7d45dee3d2e56a7d8a4ae56820f04d, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=db6e510d903e066fe70066967a0093e2, UNASSIGN}] 2023-03-24 21:55:05,719 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=db6e510d903e066fe70066967a0093e2, UNASSIGN 2023-03-24 21:55:05,719 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd7d45dee3d2e56a7d8a4ae56820f04d, UNASSIGN 2023-03-24 21:55:05,719 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=80c7f51f73f3bd5497dc15e88fadefa2, UNASSIGN 2023-03-24 21:55:05,719 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=db6e510d903e066fe70066967a0093e2, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:05,719 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=80c7f51f73f3bd5497dc15e88fadefa2, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:05,719 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=dd7d45dee3d2e56a7d8a4ae56820f04d, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:05,719 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694905719"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694905719"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694905719"}]},"ts":"1679694905719"} 2023-03-24 21:55:05,720 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694905719"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694905719"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694905719"}]},"ts":"1679694905719"} 2023-03-24 21:55:05,719 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694905719"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694905719"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694905719"}]},"ts":"1679694905719"} 2023-03-24 21:55:05,720 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=91, state=RUNNABLE; CloseRegionProcedure db6e510d903e066fe70066967a0093e2, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:05,721 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure dd7d45dee3d2e56a7d8a4ae56820f04d, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:05,721 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=89, state=RUNNABLE; CloseRegionProcedure 80c7f51f73f3bd5497dc15e88fadefa2, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:05,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-24 21:55:05,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:05,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing db6e510d903e066fe70066967a0093e2, disabling compactions & flushes 2023-03-24 21:55:05,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:05,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:05,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. after waiting 0 ms 2023-03-24 21:55:05,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:05,875 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-24 21:55:05,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:05,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:55:05,878 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:05,878 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:55:05,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 80c7f51f73f3bd5497dc15e88fadefa2, disabling compactions & flushes 2023-03-24 21:55:05,878 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:05,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:05,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. after waiting 0 ms 2023-03-24 21:55:05,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:05,878 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=db6e510d903e066fe70066967a0093e2, regionState=CLOSED 2023-03-24 21:55:05,878 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694905878"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694905878"}]},"ts":"1679694905878"} 2023-03-24 21:55:05,881 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=91 2023-03-24 21:55:05,881 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=91, state=SUCCESS; CloseRegionProcedure db6e510d903e066fe70066967a0093e2, server=jenkins-hbase9.apache.org,41807,1679694819103 in 160 msec 2023-03-24 21:55:05,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-24 21:55:05,882 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=db6e510d903e066fe70066967a0093e2, UNASSIGN in 164 msec 2023-03-24 21:55:05,883 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:05,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:55:05,884 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:55:05,884 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:05,884 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing dd7d45dee3d2e56a7d8a4ae56820f04d, disabling compactions & flushes 2023-03-24 21:55:05,884 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:05,884 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:05,884 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. after waiting 0 ms 2023-03-24 21:55:05,884 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:05,884 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=80c7f51f73f3bd5497dc15e88fadefa2, regionState=CLOSED 2023-03-24 21:55:05,884 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694905884"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694905884"}]},"ts":"1679694905884"} 2023-03-24 21:55:05,887 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=89 2023-03-24 21:55:05,887 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=89, state=SUCCESS; CloseRegionProcedure 80c7f51f73f3bd5497dc15e88fadefa2, server=jenkins-hbase9.apache.org,41807,1679694819103 in 165 msec 2023-03-24 21:55:05,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=80c7f51f73f3bd5497dc15e88fadefa2, UNASSIGN in 170 msec 2023-03-24 21:55:05,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-24 21:55:05,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:05,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:55:05,890 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:05,890 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=dd7d45dee3d2e56a7d8a4ae56820f04d, regionState=CLOSED 2023-03-24 21:55:05,891 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694905890"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694905890"}]},"ts":"1679694905890"} 2023-03-24 21:55:05,893 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=90 2023-03-24 21:55:05,893 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=90, state=SUCCESS; CloseRegionProcedure dd7d45dee3d2e56a7d8a4ae56820f04d, server=jenkins-hbase9.apache.org,41807,1679694819103 in 171 msec 2023-03-24 21:55:05,894 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=88 2023-03-24 21:55:05,894 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd7d45dee3d2e56a7d8a4ae56820f04d, UNASSIGN in 176 msec 2023-03-24 21:55:05,894 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694905894"}]},"ts":"1679694905894"} 2023-03-24 21:55:05,895 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-03-24 21:55:05,898 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-03-24 21:55:05,899 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 186 msec 2023-03-24 21:55:06,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-24 21:55:06,017 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-03-24 21:55:06,028 INFO [Listener at localhost/39907] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1679694898013-1679694906028 2023-03-24 21:55:06,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1580): Client=jenkins//172.31.2.10 snapshot request for:{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-24 21:55:06,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1679694906029 (current time:1679694906029). 2023-03-24 21:55:06,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-24 21:55:06,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] zookeeper.ReadOnlyZKClient(139): Connect 0x7e7d2180 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:55:06,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@789f2c5d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:55:06,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:55:06,037 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:39834, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:55:06,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7e7d2180 to 127.0.0.1:54680 2023-03-24 21:55:06,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:06,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-24 21:55:06,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-24 21:55:06,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-24 21:55:06,041 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-24 21:55:06,042 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1679695506041 2023-03-24 21:55:06,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-24 21:55:06,042 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1679694898013-1679694906028 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-24 21:55:06,042 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:06,042 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-24 21:55:06,142 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-24 21:55:06,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-24 21:55:06,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-24 21:55:06,143 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-24 21:55:06,343 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-24 21:55:06,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-24 21:55:06,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-24 21:55:06,345 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-24 21:55:06,448 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-03-24 21:55:06,450 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-24 21:55:06,450 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-24 21:55:06,450 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-24 21:55:06,450 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-24 21:55:06,450 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-24 21:55:06,450 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-24 21:55:06,450 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-24 21:55:06,450 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-24 21:55:06,452 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/313a08f3040d4b1a82a70be8be3a3d51_SeqId_29_, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/d837959eb97042db907d47983a1ad1f9] hfiles 2023-03-24 21:55:06,452 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/313a08f3040d4b1a82a70be8be3a3d51_SeqId_29_ 2023-03-24 21:55:06,452 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/d837959eb97042db907d47983a1ad1f9 2023-03-24 21:55:06,452 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/71d0651323034e5eac256c0f5f19f013_SeqId_29_, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c064c18447d14ed7a695f7968924001a] hfiles 2023-03-24 21:55:06,452 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/71d0651323034e5eac256c0f5f19f013_SeqId_29_ 2023-03-24 21:55:06,453 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/4231f470a5304731954c12ad8206e2b1, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/9179e35c711e43a4ac15fd4ac2a8f01b_SeqId_29_] hfiles 2023-03-24 21:55:06,453 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c064c18447d14ed7a695f7968924001a 2023-03-24 21:55:06,453 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/4231f470a5304731954c12ad8206e2b1 2023-03-24 21:55:06,453 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324c43eee7499b548a08101026b2071f356, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324780fde5425b04acdb90ab96751a03e42, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324ca2c9bc1a3594a67a55d343ad56c7a79] hfiles 2023-03-24 21:55:06,453 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/9179e35c711e43a4ac15fd4ac2a8f01b_SeqId_29_ 2023-03-24 21:55:06,453 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324c43eee7499b548a08101026b2071f356 2023-03-24 21:55:06,453 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324780fde5425b04acdb90ab96751a03e42 2023-03-24 21:55:06,453 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324ca2c9bc1a3594a67a55d343ad56c7a79 2023-03-24 21:55:06,453 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/ca69b78c3ca24abd91a01e821b6d1cec] hfiles 2023-03-24 21:55:06,453 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/ca69b78c3ca24abd91a01e821b6d1cec 2023-03-24 21:55:06,453 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/3a0abc512f3c40e4bb0b098a5d5a6367] hfiles 2023-03-24 21:55:06,453 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/3a0abc512f3c40e4bb0b098a5d5a6367 2023-03-24 21:55:06,454 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/240fee55058242128795398a51ecd239] hfiles 2023-03-24 21:55:06,454 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/240fee55058242128795398a51ecd239 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303240a18c957ac64464aa452d87984ecd741, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303244f7053aa59ef4aabacfbdad9790d4b25_del, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303248f0be98635a74e8cb762dd3ac9d9dad1, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230324ae6b4dfca433486082c9e1f7e3e66627, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230324ee47c7143e3548a0946b7998233c5cb3, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032435e7af8aff134bc7978ad6fb7cd6c2be, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032436c0ae60afb848de99a9ba12d53edbbe, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324771fcbddd6164a379354a7c114971988, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324a75914ff0ba04d5ca4adc2fe4eaab8d7_del, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324ee6b06f3c7fd4a228457033777494a55, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032434f646a8dbf04661a5b61817f4fadd69, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303243822b4be036143d79f8833a8e23ac8ec_del, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032448d400f1b8b64d3b8f83bec372eef9b3, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303245040c195a5574df4a60c60e92e90ed3a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230324e32db202e5394db1980382e90a3489c3] hfiles 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303240a18c957ac64464aa452d87984ecd741 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303244f7053aa59ef4aabacfbdad9790d4b25_del 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303248f0be98635a74e8cb762dd3ac9d9dad1 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230324ae6b4dfca433486082c9e1f7e3e66627 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230324ee47c7143e3548a0946b7998233c5cb3 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032435e7af8aff134bc7978ad6fb7cd6c2be 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032436c0ae60afb848de99a9ba12d53edbbe 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324771fcbddd6164a379354a7c114971988 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324a75914ff0ba04d5ca4adc2fe4eaab8d7_del 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324ee6b06f3c7fd4a228457033777494a55 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032434f646a8dbf04661a5b61817f4fadd69 2023-03-24 21:55:06,466 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303243822b4be036143d79f8833a8e23ac8ec_del 2023-03-24 21:55:06,467 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032448d400f1b8b64d3b8f83bec372eef9b3 2023-03-24 21:55:06,467 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303245040c195a5574df4a60c60e92e90ed3a 2023-03-24 21:55:06,467 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230324e32db202e5394db1980382e90a3489c3 2023-03-24 21:55:06,645 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-24 21:55:06,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-24 21:55:06,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-24 21:55:06,646 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-24 21:55:06,871 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-03-24 21:55:06,871 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1679694898013-1679694906028 2023-03-24 21:55:06,872 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679694898013-1679694906028 2023-03-24 21:55:07,146 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-24 21:55:07,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-24 21:55:07,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-24 21:55:07,147 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-24 21:55:07,298 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679694898013-1679694906028 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.hbase-snapshot/hbase-failsafe-snaptb-1679694898013-1679694906028 2023-03-24 21:55:08,113 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1679694898013-1679694906028 of table testCompactionWithHFileLink completed 2023-03-24 21:55:08,113 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679694898013-1679694906028 2023-03-24 21:55:08,113 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679694898013-1679694906028 2023-03-24 21:55:08,113 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1679694898013-1679694906028 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1679694906042Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1679694906448 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1679694898013-1679694906028 at 1679694906871 (+423 ms)Loading Region manifests for hbase-failsafe-snaptb-1679694898013-1679694906028 at 1679694906871Writing data manifest for hbase-failsafe-snaptb-1679694898013-1679694906028 at 1679694906874 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1679694898013-1679694906028 at 1679694907280 (+406 ms)Snapshot hbase-failsafe-snaptb-1679694898013-1679694906028 of table testCompactionWithHFileLink completed at 1679694908113 (+833 ms) 2023-03-24 21:55:08,114 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-24 21:55:08,115 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0740 sec 2023-03-24 21:55:08,148 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-24 21:55:08,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-24 21:55:08,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1679694898013-1679694906028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-24 21:55:08,294 INFO [regionserver/jenkins-hbase9:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-03-24 21:55:08,294 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:55:08,294 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6c0e0f2d to 127.0.0.1:54680 2023-03-24 21:55:08,294 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:08,295 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:55:08,295 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x28a98f9d to 127.0.0.1:54680 2023-03-24 21:55:08,295 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:08,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1679694898013" table: "testCompactionWithHFileLink" creation_time: 1679694898027 type: FLUSH version: 2 ttl: 0 ) 2023-03-24 21:55:08,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1679694898013 as table=testCompactionWithHFileLink 2023-03-24 21:55:08,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-24 21:55:08,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-24 21:55:08,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-24 21:55:08,747 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-24 21:55:08,748 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-24 21:55:08,748 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-24 21:55:08,749 INFO [PEWorker-2] procedure.RestoreSnapshotProcedure(384): Starting restore snapshot={ ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-24 21:55:08,753 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1679694898013" table: "testCompactionWithHFileLink" creation_time: 1679694898027 type: FLUSH version: 2 ttl: 0 2023-03-24 21:55:08,754 DEBUG [PEWorker-2] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink 2023-03-24 21:55:08,759 DEBUG [PEWorker-2] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-03-24 21:55:08,762 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324c43eee7499b548a08101026b2071f356 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230324c43eee7499b548a08101026b2071f356 2023-03-24 21:55:08,763 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324ca2c9bc1a3594a67a55d343ad56c7a79 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324ca2c9bc1a3594a67a55d343ad56c7a79 2023-03-24 21:55:08,764 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324780fde5425b04acdb90ab96751a03e42 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324780fde5425b04acdb90ab96751a03e42 2023-03-24 21:55:08,764 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303242e179998f2e24897b7653543ea105766 of snapshot=snaptb-1679694898013 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-24 21:55:08,767 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303249102367c04aa4f6681d55ff092535183 of snapshot=snaptb-1679694898013 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-24 21:55:08,770 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230324d7da3a5c947a4742b6e70403b0b8915b of snapshot=snaptb-1679694898013 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-24 21:55:08,773 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230324d7fcdeeeaa884fb399b7825fb7a06335 of snapshot=snaptb-1679694898013 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-24 21:55:08,775 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023032408ac34e9f9cf436e9e176b1f391cb5cf of snapshot=snaptb-1679694898013 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-24 21:55:08,778 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303240938f5625b8f4728ae55dff1564edffb of snapshot=snaptb-1679694898013 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-24 21:55:08,780 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230324ab93dd4f06c644be89323eed2735e5f0 of snapshot=snaptb-1679694898013 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-24 21:55:08,783 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230324dfed51c8c5474a239a39070aba3b8b49 of snapshot=snaptb-1679694898013 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-24 21:55:08,785 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023032438af3f7a90284a219128aa718c8b109e of snapshot=snaptb-1679694898013 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-24 21:55:08,788 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230324af079d2fd1914f4fafbcb41bd9adc5b1 of snapshot=snaptb-1679694898013 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-24 21:55:08,790 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230324d0c5015a0307438da385076af4d9d54f of snapshot=snaptb-1679694898013 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-24 21:55:08,793 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230324ea6e832756384abbbdebb994807e4349 of snapshot=snaptb-1679694898013 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-24 21:55:08,798 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303243822b4be036143d79f8833a8e23ac8ec_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303243822b4be036143d79f8833a8e23ac8ec_del 2023-03-24 21:55:08,799 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324a75914ff0ba04d5ca4adc2fe4eaab8d7_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230324a75914ff0ba04d5ca4adc2fe4eaab8d7_del 2023-03-24 21:55:08,801 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303244f7053aa59ef4aabacfbdad9790d4b25_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303244f7053aa59ef4aabacfbdad9790d4b25_del 2023-03-24 21:55:08,801 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:55:08,801 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:08,801 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:08,804 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/d837959eb97042db907d47983a1ad1f9 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/d837959eb97042db907d47983a1ad1f9 2023-03-24 21:55:08,804 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c064c18447d14ed7a695f7968924001a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/c064c18447d14ed7a695f7968924001a 2023-03-24 21:55:08,805 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/4231f470a5304731954c12ad8206e2b1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/4231f470a5304731954c12ad8206e2b1 2023-03-24 21:55:08,806 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/71d0651323034e5eac256c0f5f19f013_SeqId_29_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/71d0651323034e5eac256c0f5f19f013_SeqId_29_ 2023-03-24 21:55:08,806 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8dbaeb15a2174a16b5567bf58b9bbecd of snapshot=snaptb-1679694898013 to region=db6e510d903e066fe70066967a0093e2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,806 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/9179e35c711e43a4ac15fd4ac2a8f01b_SeqId_29_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/9179e35c711e43a4ac15fd4ac2a8f01b_SeqId_29_ 2023-03-24 21:55:08,806 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d27247c35aab4a5e95522f4b4082ee48 of snapshot=snaptb-1679694898013 to region=dd7d45dee3d2e56a7d8a4ae56820f04d table=testCompactionWithHFileLink 2023-03-24 21:55:08,806 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/313a08f3040d4b1a82a70be8be3a3d51_SeqId_29_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/313a08f3040d4b1a82a70be8be3a3d51_SeqId_29_ 2023-03-24 21:55:08,806 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cd41d6bc7625452d8cd50d0509b4a453 of snapshot=snaptb-1679694898013 to region=80c7f51f73f3bd5497dc15e88fadefa2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,813 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0ade5f95824844f1bdb3bf6657afc66c of snapshot=snaptb-1679694898013 to region=db6e510d903e066fe70066967a0093e2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,813 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 57f179ca64e64316ba7cd9ca99b22557 of snapshot=snaptb-1679694898013 to region=80c7f51f73f3bd5497dc15e88fadefa2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,814 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3a255a9bd6b34f9e82ea9b25a9f3431b of snapshot=snaptb-1679694898013 to region=dd7d45dee3d2e56a7d8a4ae56820f04d table=testCompactionWithHFileLink 2023-03-24 21:55:08,816 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c2c3a5c11515473ba6a2ac8f75a53ddb of snapshot=snaptb-1679694898013 to region=db6e510d903e066fe70066967a0093e2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,821 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 00999c369b14425ea335da2d7d948d08 of snapshot=snaptb-1679694898013 to region=80c7f51f73f3bd5497dc15e88fadefa2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,822 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d9d7de1995344cbab674aca5eed6eee7 of snapshot=snaptb-1679694898013 to region=dd7d45dee3d2e56a7d8a4ae56820f04d table=testCompactionWithHFileLink 2023-03-24 21:55:08,823 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 79acd90194484e429e64a28f2b3d5703 of snapshot=snaptb-1679694898013 to region=db6e510d903e066fe70066967a0093e2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,827 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 343affb19b444075983c87a317f71c54 of snapshot=snaptb-1679694898013 to region=80c7f51f73f3bd5497dc15e88fadefa2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,827 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 502a23eefb564b108fd0804dd81ccc60 of snapshot=snaptb-1679694898013 to region=dd7d45dee3d2e56a7d8a4ae56820f04d table=testCompactionWithHFileLink 2023-03-24 21:55:08,830 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/3a0abc512f3c40e4bb0b098a5d5a6367 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/3a0abc512f3c40e4bb0b098a5d5a6367 2023-03-24 21:55:08,830 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9430792f9a7c45b2b628cf070eabbb41 of snapshot=snaptb-1679694898013 to region=db6e510d903e066fe70066967a0093e2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,832 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/ca69b78c3ca24abd91a01e821b6d1cec to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/ca69b78c3ca24abd91a01e821b6d1cec 2023-03-24 21:55:08,832 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b0f3a1484555406892bf1e9f9963292d of snapshot=snaptb-1679694898013 to region=80c7f51f73f3bd5497dc15e88fadefa2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,832 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/240fee55058242128795398a51ecd239 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/240fee55058242128795398a51ecd239 2023-03-24 21:55:08,832 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 265c8226a3a1414583f189917e8a53c7 of snapshot=snaptb-1679694898013 to region=dd7d45dee3d2e56a7d8a4ae56820f04d table=testCompactionWithHFileLink 2023-03-24 21:55:08,833 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0eb237e6e4534356b0c3eb93951ca94b of snapshot=snaptb-1679694898013 to region=db6e510d903e066fe70066967a0093e2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,835 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7e5ea937345a494e87b1c26c366f8cd7 of snapshot=snaptb-1679694898013 to region=80c7f51f73f3bd5497dc15e88fadefa2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,835 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink aad477eb45dc4b8fbb0b9c5cc5ab0982 of snapshot=snaptb-1679694898013 to region=dd7d45dee3d2e56a7d8a4ae56820f04d table=testCompactionWithHFileLink 2023-03-24 21:55:08,836 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6b11f68343084dc3bc8a631c212cf15c of snapshot=snaptb-1679694898013 to region=db6e510d903e066fe70066967a0093e2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,838 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a87ee0a5f4f1401a967dae27a4650f1a of snapshot=snaptb-1679694898013 to region=80c7f51f73f3bd5497dc15e88fadefa2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,839 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ff2402da2b7244269d37afebd5825ed5 of snapshot=snaptb-1679694898013 to region=dd7d45dee3d2e56a7d8a4ae56820f04d table=testCompactionWithHFileLink 2023-03-24 21:55:08,840 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9f76b2a9f280438bb6a623955f71adae of snapshot=snaptb-1679694898013 to region=db6e510d903e066fe70066967a0093e2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,841 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0beec99d01ea409ab5eb29b007b9a14f of snapshot=snaptb-1679694898013 to region=80c7f51f73f3bd5497dc15e88fadefa2 table=testCompactionWithHFileLink 2023-03-24 21:55:08,842 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 73842ebef8cc4e218b9f1e78ba7eead2 of snapshot=snaptb-1679694898013 to region=dd7d45dee3d2e56a7d8a4ae56820f04d table=testCompactionWithHFileLink 2023-03-24 21:55:08,845 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1679694898013" table: "testCompactionWithHFileLink" creation_time: 1679694898027 type: FLUSH version: 2 ttl: 0 2023-03-24 21:55:08,846 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679694908846"}]},"ts":"9223372036854775807"} 2023-03-24 21:55:08,846 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679694908846"}]},"ts":"9223372036854775807"} 2023-03-24 21:55:08,846 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679694908846"}]},"ts":"9223372036854775807"} 2023-03-24 21:55:08,848 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-03-24 21:55:08,848 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 80c7f51f73f3bd5497dc15e88fadefa2, NAME => 'testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => db6e510d903e066fe70066967a0093e2, NAME => 'testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => dd7d45dee3d2e56a7d8a4ae56820f04d, NAME => 'testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.', STARTKEY => '1', ENDKEY => '2'}] 2023-03-24 21:55:08,849 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694908847"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694908847"}]},"ts":"1679694908847"} 2023-03-24 21:55:08,849 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694908847"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694908847"}]},"ts":"1679694908847"} 2023-03-24 21:55:08,849 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694908847"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694908847"}]},"ts":"1679694908847"} 2023-03-24 21:55:08,850 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-24 21:55:08,850 INFO [PEWorker-2] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-03-24 21:55:08,850 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 80c7f51f73f3bd5497dc15e88fadefa2, NAME => 'testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => db6e510d903e066fe70066967a0093e2, NAME => 'testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => dd7d45dee3d2e56a7d8a4ae56820f04d, NAME => 'testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.', STARTKEY => '1', ENDKEY => '2'}] 2023-03-24 21:55:08,851 INFO [PEWorker-2] procedure.RestoreSnapshotProcedure(457): Restore snapshot={ ss=snaptb-1679694898013 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-03-24 21:55:08,853 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1679694898013" table: "testCompactionWithHFileLink" creation_time: 1679694898027 type: FLUSH version: 2 ttl: 0 ) in 526 msec 2023-03-24 21:55:08,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-24 21:55:08,944 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-03-24 21:55:08,944 INFO [Listener at localhost/39907] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1679694898013-1679694906028 2023-03-24 21:55:08,948 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(748): Client=jenkins//172.31.2.10 delete name: "hbase-failsafe-snaptb-1679694898013-1679694906028" 2023-03-24 21:55:08,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1679694898013-1679694906028 2023-03-24 21:55:08,953 INFO [Listener at localhost/39907] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-03-24 21:55:08,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$10(2385): Client=jenkins//172.31.2.10 enable testCompactionWithHFileLink 2023-03-24 21:55:08,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-03-24 21:55:08,965 INFO [PEWorker-5] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-03-24 21:55:08,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-24 21:55:08,965 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694908965"}]},"ts":"1679694908965"} 2023-03-24 21:55:08,967 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-24 21:55:08,970 DEBUG [PEWorker-5] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-03-24 21:55:08,970 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=80c7f51f73f3bd5497dc15e88fadefa2, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd7d45dee3d2e56a7d8a4ae56820f04d, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=db6e510d903e066fe70066967a0093e2, ASSIGN}] 2023-03-24 21:55:08,971 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd7d45dee3d2e56a7d8a4ae56820f04d, ASSIGN 2023-03-24 21:55:08,972 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=db6e510d903e066fe70066967a0093e2, ASSIGN 2023-03-24 21:55:08,972 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=80c7f51f73f3bd5497dc15e88fadefa2, ASSIGN 2023-03-24 21:55:08,972 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd7d45dee3d2e56a7d8a4ae56820f04d, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-24 21:55:08,972 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=80c7f51f73f3bd5497dc15e88fadefa2, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-24 21:55:08,972 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=db6e510d903e066fe70066967a0093e2, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-24 21:55:09,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-24 21:55:09,124 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=dd7d45dee3d2e56a7d8a4ae56820f04d, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:09,124 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=80c7f51f73f3bd5497dc15e88fadefa2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:09,124 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=db6e510d903e066fe70066967a0093e2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:09,124 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694909124"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694909124"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694909124"}]},"ts":"1679694909124"} 2023-03-24 21:55:09,124 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694909124"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694909124"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694909124"}]},"ts":"1679694909124"} 2023-03-24 21:55:09,124 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694909124"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694909124"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694909124"}]},"ts":"1679694909124"} 2023-03-24 21:55:09,125 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=99, state=RUNNABLE; OpenRegionProcedure dd7d45dee3d2e56a7d8a4ae56820f04d, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:09,126 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure 80c7f51f73f3bd5497dc15e88fadefa2, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:09,127 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=100, state=RUNNABLE; OpenRegionProcedure db6e510d903e066fe70066967a0093e2, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:09,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-24 21:55:09,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:09,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 80c7f51f73f3bd5497dc15e88fadefa2, NAME => 'testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.', STARTKEY => '', ENDKEY => '1'} 2023-03-24 21:55:09,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:55:09,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:09,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:55:09,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:55:09,281 INFO [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:55:09,282 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1 2023-03-24 21:55:09,282 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1 2023-03-24 21:55:09,283 INFO [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80c7f51f73f3bd5497dc15e88fadefa2 columnFamilyName family1 2023-03-24 21:55:09,293 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-00999c369b14425ea335da2d7d948d08 2023-03-24 21:55:09,299 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-343affb19b444075983c87a317f71c54 2023-03-24 21:55:09,303 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-57f179ca64e64316ba7cd9ca99b22557 2023-03-24 21:55:09,307 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-cd41d6bc7625452d8cd50d0509b4a453 2023-03-24 21:55:09,308 INFO [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(310): Store=80c7f51f73f3bd5497dc15e88fadefa2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:09,308 INFO [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:55:09,309 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2 2023-03-24 21:55:09,309 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2 2023-03-24 21:55:09,309 INFO [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80c7f51f73f3bd5497dc15e88fadefa2 columnFamilyName family2 2023-03-24 21:55:09,315 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-0beec99d01ea409ab5eb29b007b9a14f 2023-03-24 21:55:09,319 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-7e5ea937345a494e87b1c26c366f8cd7 2023-03-24 21:55:09,323 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-a87ee0a5f4f1401a967dae27a4650f1a 2023-03-24 21:55:09,328 DEBUG [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-b0f3a1484555406892bf1e9f9963292d 2023-03-24 21:55:09,328 INFO [StoreOpener-80c7f51f73f3bd5497dc15e88fadefa2-1] regionserver.HStore(310): Store=80c7f51f73f3bd5497dc15e88fadefa2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:09,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:55:09,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:55:09,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:55:09,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:55:09,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 80c7f51f73f3bd5497dc15e88fadefa2; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73301353, jitterRate=0.09227527678012848}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:55:09,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:55:09,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2., pid=102, masterSystemTime=1679694909277 2023-03-24 21:55:09,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:55:09,334 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-24 21:55:09,334 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-24 21:55:09,334 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-24 21:55:09,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:09,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:09,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:09,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dd7d45dee3d2e56a7d8a4ae56820f04d, NAME => 'testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.', STARTKEY => '1', ENDKEY => '2'} 2023-03-24 21:55:09,336 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=80c7f51f73f3bd5497dc15e88fadefa2, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:09,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:09,336 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694909336"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694909336"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694909336"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694909336"}]},"ts":"1679694909336"} 2023-03-24 21:55:09,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:09,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:09,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:09,336 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-24 21:55:09,336 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1912): 80c7f51f73f3bd5497dc15e88fadefa2/family1 is initiating minor compaction (all files) 2023-03-24 21:55:09,336 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-24 21:55:09,336 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 80c7f51f73f3bd5497dc15e88fadefa2/family1 in testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:09,337 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HStore(1912): 80c7f51f73f3bd5497dc15e88fadefa2/family2 is initiating minor compaction (all files) 2023-03-24 21:55:09,337 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 80c7f51f73f3bd5497dc15e88fadefa2/family2 in testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:09,337 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-cd41d6bc7625452d8cd50d0509b4a453, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-57f179ca64e64316ba7cd9ca99b22557, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-00999c369b14425ea335da2d7d948d08, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-343affb19b444075983c87a317f71c54] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp, totalSize=21.6 K 2023-03-24 21:55:09,337 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-b0f3a1484555406892bf1e9f9963292d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-7e5ea937345a494e87b1c26c366f8cd7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-a87ee0a5f4f1401a967dae27a4650f1a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-0beec99d01ea409ab5eb29b007b9a14f] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp, totalSize=20.4 K 2023-03-24 21:55:09,337 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-cd41d6bc7625452d8cd50d0509b4a453, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694892684 2023-03-24 21:55:09,337 INFO [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:09,337 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-b0f3a1484555406892bf1e9f9963292d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694892684 2023-03-24 21:55:09,338 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-57f179ca64e64316ba7cd9ca99b22557, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694893924 2023-03-24 21:55:09,338 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-7e5ea937345a494e87b1c26c366f8cd7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694893924 2023-03-24 21:55:09,338 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1 2023-03-24 21:55:09,339 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1 2023-03-24 21:55:09,339 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-00999c369b14425ea335da2d7d948d08, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694894625 2023-03-24 21:55:09,339 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-a87ee0a5f4f1401a967dae27a4650f1a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694894625 2023-03-24 21:55:09,339 INFO [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd7d45dee3d2e56a7d8a4ae56820f04d columnFamilyName family1 2023-03-24 21:55:09,339 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-343affb19b444075983c87a317f71c54, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694896541 2023-03-24 21:55:09,339 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-0beec99d01ea409ab5eb29b007b9a14f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694896541 2023-03-24 21:55:09,343 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=98 2023-03-24 21:55:09,344 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure 80c7f51f73f3bd5497dc15e88fadefa2, server=jenkins-hbase9.apache.org,41807,1679694819103 in 216 msec 2023-03-24 21:55:09,345 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=80c7f51f73f3bd5497dc15e88fadefa2, ASSIGN in 374 msec 2023-03-24 21:55:09,359 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-3a255a9bd6b34f9e82ea9b25a9f3431b 2023-03-24 21:55:09,367 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-502a23eefb564b108fd0804dd81ccc60 2023-03-24 21:55:09,390 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-d27247c35aab4a5e95522f4b4082ee48 2023-03-24 21:55:09,395 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] throttle.PressureAwareThroughputController(145): 80c7f51f73f3bd5497dc15e88fadefa2#family2#compaction#113 average throughput is 0.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 65.38 MB/second 2023-03-24 21:55:09,404 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): 80c7f51f73f3bd5497dc15e88fadefa2#family1#compaction#114 average throughput is 0.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-03-24 21:55:09,406 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-d9d7de1995344cbab674aca5eed6eee7 2023-03-24 21:55:09,406 INFO [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(310): Store=dd7d45dee3d2e56a7d8a4ae56820f04d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:09,406 INFO [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:09,407 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2 2023-03-24 21:55:09,408 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2 2023-03-24 21:55:09,408 INFO [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd7d45dee3d2e56a7d8a4ae56820f04d columnFamilyName family2 2023-03-24 21:55:09,416 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-265c8226a3a1414583f189917e8a53c7 2023-03-24 21:55:09,421 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-73842ebef8cc4e218b9f1e78ba7eead2 2023-03-24 21:55:09,425 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-aad477eb45dc4b8fbb0b9c5cc5ab0982 2023-03-24 21:55:09,430 DEBUG [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-ff2402da2b7244269d37afebd5825ed5 2023-03-24 21:55:09,430 INFO [StoreOpener-dd7d45dee3d2e56a7d8a4ae56820f04d-1] regionserver.HStore(310): Store=dd7d45dee3d2e56a7d8a4ae56820f04d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:09,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:09,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:09,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:55:09,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:09,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened dd7d45dee3d2e56a7d8a4ae56820f04d; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62071729, jitterRate=-0.07505916059017181}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:55:09,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:55:09,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d., pid=101, masterSystemTime=1679694909277 2023-03-24 21:55:09,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-24 21:55:09,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-24 21:55:09,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:09,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:09,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:09,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => db6e510d903e066fe70066967a0093e2, NAME => 'testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.', STARTKEY => '2', ENDKEY => ''} 2023-03-24 21:55:09,437 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=dd7d45dee3d2e56a7d8a4ae56820f04d, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:09,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:09,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:09,438 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694909437"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694909437"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694909437"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694909437"}]},"ts":"1679694909437"} 2023-03-24 21:55:09,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:09,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:09,439 INFO [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:09,440 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1 2023-03-24 21:55:09,440 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1 2023-03-24 21:55:09,440 INFO [StoreOpener-db6e510d903e066fe70066967a0093e2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db6e510d903e066fe70066967a0093e2 columnFamilyName family1 2023-03-24 21:55:09,440 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=99 2023-03-24 21:55:09,440 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=99, state=SUCCESS; OpenRegionProcedure dd7d45dee3d2e56a7d8a4ae56820f04d, server=jenkins-hbase9.apache.org,41807,1679694819103 in 314 msec 2023-03-24 21:55:09,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd7d45dee3d2e56a7d8a4ae56820f04d, ASSIGN in 470 msec 2023-03-24 21:55:09,449 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-0ade5f95824844f1bdb3bf6657afc66c 2023-03-24 21:55:09,453 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-79acd90194484e429e64a28f2b3d5703 2023-03-24 21:55:09,457 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-8dbaeb15a2174a16b5567bf58b9bbecd 2023-03-24 21:55:09,467 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-c2c3a5c11515473ba6a2ac8f75a53ddb 2023-03-24 21:55:09,467 INFO [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(310): Store=db6e510d903e066fe70066967a0093e2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:09,467 INFO [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:09,468 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2 2023-03-24 21:55:09,468 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2 2023-03-24 21:55:09,469 INFO [StoreOpener-db6e510d903e066fe70066967a0093e2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db6e510d903e066fe70066967a0093e2 columnFamilyName family2 2023-03-24 21:55:09,478 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-0eb237e6e4534356b0c3eb93951ca94b 2023-03-24 21:55:09,484 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-6b11f68343084dc3bc8a631c212cf15c 2023-03-24 21:55:09,490 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-9430792f9a7c45b2b628cf070eabbb41 2023-03-24 21:55:09,495 DEBUG [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(539): loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-9f76b2a9f280438bb6a623955f71adae 2023-03-24 21:55:09,495 INFO [StoreOpener-db6e510d903e066fe70066967a0093e2-1] regionserver.HStore(310): Store=db6e510d903e066fe70066967a0093e2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:09,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:09,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:09,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:55:09,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:09,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened db6e510d903e066fe70066967a0093e2; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67344465, jitterRate=0.0035107284784317017}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:55:09,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:55:09,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2., pid=103, masterSystemTime=1679694909277 2023-03-24 21:55:09,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-24 21:55:09,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-24 21:55:09,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:09,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:09,502 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=db6e510d903e066fe70066967a0093e2, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:09,502 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679694909502"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694909502"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694909502"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694909502"}]},"ts":"1679694909502"} 2023-03-24 21:55:09,504 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=100 2023-03-24 21:55:09,504 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=100, state=SUCCESS; OpenRegionProcedure db6e510d903e066fe70066967a0093e2, server=jenkins-hbase9.apache.org,41807,1679694819103 in 376 msec 2023-03-24 21:55:09,505 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-03-24 21:55:09,505 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=db6e510d903e066fe70066967a0093e2, ASSIGN in 534 msec 2023-03-24 21:55:09,506 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694909506"}]},"ts":"1679694909506"} 2023-03-24 21:55:09,506 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-24 21:55:09,508 INFO [PEWorker-2] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-03-24 21:55:09,509 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 550 msec 2023-03-24 21:55:09,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-24 21:55:09,569 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-03-24 21:55:09,580 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:09,586 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:09,595 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:09,596 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:09,602 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-24 21:55:09,615 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-24 21:55:09,616 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:09,616 DEBUG [Listener at localhost/39907] zookeeper.ReadOnlyZKClient(139): Connect 0x62df6183 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:55:09,620 DEBUG [Listener at localhost/39907] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2ab41399, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:55:09,620 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230324 for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:09,621 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230324 for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:09,621 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230324 for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:10,100 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324eac415179d164bdbb5030b5da789e4a4 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230324eac415179d164bdbb5030b5da789e4a4 2023-03-24 21:55:10,101 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230324 table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:10,113 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:55:10,117 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:39840, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:55:10,124 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:55:10,126 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:53202, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:55:10,134 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230324/family1/cafa1d24c3e445a7bd77472e5b03f396 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-03-24 21:55:10,135 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230324/family1/cafa1d24c3e445a7bd77472e5b03f396}] 2023-03-24 21:55:10,139 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230324/family1/cafa1d24c3e445a7bd77472e5b03f396 for inclusion in db6e510d903e066fe70066967a0093e2/family1 2023-03-24 21:55:10,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-03-24 21:55:10,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-24 21:55:10,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:55:10,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230324/family1/cafa1d24c3e445a7bd77472e5b03f396 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__t0do05pgnt0jg2em1fh5ec2inmapd85ajea0hmd7vjalchv1e7bd573qa65qrh3d/family1/cafa1d24c3e445a7bd77472e5b03f396 2023-03-24 21:55:10,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__t0do05pgnt0jg2em1fh5ec2inmapd85ajea0hmd7vjalchv1e7bd573qa65qrh3d/family1/cafa1d24c3e445a7bd77472e5b03f396 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/4627c5b717b3497a95bbf1567a1e407e_SeqId_34_ 2023-03-24 21:55:10,144 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__t0do05pgnt0jg2em1fh5ec2inmapd85ajea0hmd7vjalchv1e7bd573qa65qrh3d/family1/cafa1d24c3e445a7bd77472e5b03f396 into db6e510d903e066fe70066967a0093e2/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/4627c5b717b3497a95bbf1567a1e407e_SeqId_34_ - updating store file list. 2023-03-24 21:55:10,147 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/4627c5b717b3497a95bbf1567a1e407e_SeqId_34_ into db6e510d903e066fe70066967a0093e2/family1 2023-03-24 21:55:10,148 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__t0do05pgnt0jg2em1fh5ec2inmapd85ajea0hmd7vjalchv1e7bd573qa65qrh3d/family1/cafa1d24c3e445a7bd77472e5b03f396 into db6e510d903e066fe70066967a0093e2/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/4627c5b717b3497a95bbf1567a1e407e_SeqId_34_) 2023-03-24 21:55:10,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__t0do05pgnt0jg2em1fh5ec2inmapd85ajea0hmd7vjalchv1e7bd573qa65qrh3d/family1/cafa1d24c3e445a7bd77472e5b03f396 2023-03-24 21:55:10,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:10,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-24 21:55:10,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(1912): db6e510d903e066fe70066967a0093e2/family1 is initiating minor compaction (all files) 2023-03-24 21:55:10,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@68e1b705; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-24 21:55:10,152 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230324 table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:10,152 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:10,154 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023032408ac34e9f9cf436e9e176b1f391cb5cf to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023032408ac34e9f9cf436e9e176b1f391cb5cf 2023-03-24 21:55:10,155 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303240938f5625b8f4728ae55dff1564edffb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303240938f5625b8f4728ae55dff1564edffb 2023-03-24 21:55:10,156 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230324ab93dd4f06c644be89323eed2735e5f0 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230324ab93dd4f06c644be89323eed2735e5f0 2023-03-24 21:55:10,156 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230324dfed51c8c5474a239a39070aba3b8b49 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230324dfed51c8c5474a239a39070aba3b8b49 2023-03-24 21:55:10,156 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-24 21:55:10,156 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230324' 2023-03-24 21:55:10,157 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230324 for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:10,495 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032409134915533e4ac4bab7440a7d50f3ed to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032409134915533e4ac4bab7440a7d50f3ed 2023-03-24 21:55:10,496 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230324 table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:10,499 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324c45c32948a5546ba80f26ce58b93238b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230324c45c32948a5546ba80f26ce58b93238b 2023-03-24 21:55:10,500 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230324 table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:10,521 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230324/family1/24799cc8876443768d46cb3c3886940b first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-03-24 21:55:10,530 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230324/family1/24799cc8876443768d46cb3c3886940b}] 2023-03-24 21:55:10,534 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230324/family1/24799cc8876443768d46cb3c3886940b for inclusion in dd7d45dee3d2e56a7d8a4ae56820f04d/family1 2023-03-24 21:55:10,535 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230324/family1/016c18c0cea34169ba379907a415e9d1 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-03-24 21:55:10,536 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=32 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230324/family1/016c18c0cea34169ba379907a415e9d1}] 2023-03-24 21:55:10,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-03-24 21:55:10,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-24 21:55:10,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:55:10,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230324/family1/24799cc8876443768d46cb3c3886940b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__hcledh90i4gttk95apqbd0b8rci5k90mh3k4gl74v68gfagt387fh0pusraqq33v/family1/24799cc8876443768d46cb3c3886940b 2023-03-24 21:55:10,538 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230324/family1/016c18c0cea34169ba379907a415e9d1 for inclusion in 80c7f51f73f3bd5497dc15e88fadefa2/family1 2023-03-24 21:55:10,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__hcledh90i4gttk95apqbd0b8rci5k90mh3k4gl74v68gfagt387fh0pusraqq33v/family1/24799cc8876443768d46cb3c3886940b as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/1e78d4ad12754b3a8b2c07974f14d019_SeqId_34_ 2023-03-24 21:55:10,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-03-24 21:55:10,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last=1 2023-03-24 21:55:10,540 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__hcledh90i4gttk95apqbd0b8rci5k90mh3k4gl74v68gfagt387fh0pusraqq33v/family1/24799cc8876443768d46cb3c3886940b into dd7d45dee3d2e56a7d8a4ae56820f04d/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/1e78d4ad12754b3a8b2c07974f14d019_SeqId_34_ - updating store file list. 2023-03-24 21:55:10,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:55:10,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230324/family1/016c18c0cea34169ba379907a415e9d1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__hqs36n7mjuevfojmrdcoeo40giitab6742lfcoq9apg3lm0r1b3tirn6fjfqret7/family1/016c18c0cea34169ba379907a415e9d1 2023-03-24 21:55:10,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__hqs36n7mjuevfojmrdcoeo40giitab6742lfcoq9apg3lm0r1b3tirn6fjfqret7/family1/016c18c0cea34169ba379907a415e9d1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/d4dd7949c86f44ddb1f91b6c668cfb33_SeqId_34_ 2023-03-24 21:55:10,543 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/1e78d4ad12754b3a8b2c07974f14d019_SeqId_34_ into dd7d45dee3d2e56a7d8a4ae56820f04d/family1 2023-03-24 21:55:10,543 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__hcledh90i4gttk95apqbd0b8rci5k90mh3k4gl74v68gfagt387fh0pusraqq33v/family1/24799cc8876443768d46cb3c3886940b into dd7d45dee3d2e56a7d8a4ae56820f04d/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/1e78d4ad12754b3a8b2c07974f14d019_SeqId_34_) 2023-03-24 21:55:10,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__hqs36n7mjuevfojmrdcoeo40giitab6742lfcoq9apg3lm0r1b3tirn6fjfqret7/family1/016c18c0cea34169ba379907a415e9d1 into 80c7f51f73f3bd5497dc15e88fadefa2/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/d4dd7949c86f44ddb1f91b6c668cfb33_SeqId_34_ - updating store file list. 2023-03-24 21:55:10,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__hcledh90i4gttk95apqbd0b8rci5k90mh3k4gl74v68gfagt387fh0pusraqq33v/family1/24799cc8876443768d46cb3c3886940b 2023-03-24 21:55:10,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:10,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-24 21:55:10,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.HStore(1912): dd7d45dee3d2e56a7d8a4ae56820f04d/family1 is initiating minor compaction (all files) 2023-03-24 21:55:10,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3b30e1dd; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-03-24 21:55:10,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/d4dd7949c86f44ddb1f91b6c668cfb33_SeqId_34_ into 80c7f51f73f3bd5497dc15e88fadefa2/family1 2023-03-24 21:55:10,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__hqs36n7mjuevfojmrdcoeo40giitab6742lfcoq9apg3lm0r1b3tirn6fjfqret7/family1/016c18c0cea34169ba379907a415e9d1 into 80c7f51f73f3bd5497dc15e88fadefa2/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/d4dd7949c86f44ddb1f91b6c668cfb33_SeqId_34_) 2023-03-24 21:55:10,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testCompactionWithHFileLink__hqs36n7mjuevfojmrdcoeo40giitab6742lfcoq9apg3lm0r1b3tirn6fjfqret7/family1/016c18c0cea34169ba379907a415e9d1 2023-03-24 21:55:10,548 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230324 table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:10,548 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230324 table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:10,549 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:10,549 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:10,550 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303242e179998f2e24897b7653543ea105766 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303242e179998f2e24897b7653543ea105766 2023-03-24 21:55:10,551 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023032438af3f7a90284a219128aa718c8b109e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023032438af3f7a90284a219128aa718c8b109e 2023-03-24 21:55:10,552 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303249102367c04aa4f6681d55ff092535183 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303249102367c04aa4f6681d55ff092535183 2023-03-24 21:55:10,552 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230324af079d2fd1914f4fafbcb41bd9adc5b1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230324af079d2fd1914f4fafbcb41bd9adc5b1 2023-03-24 21:55:10,553 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230324d7da3a5c947a4742b6e70403b0b8915b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230324d7da3a5c947a4742b6e70403b0b8915b 2023-03-24 21:55:10,554 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230324d0c5015a0307438da385076af4d9d54f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230324d0c5015a0307438da385076af4d9d54f 2023-03-24 21:55:10,554 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230324d7fcdeeeaa884fb399b7825fb7a06335 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230324d7fcdeeeaa884fb399b7825fb7a06335 2023-03-24 21:55:10,554 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-24 21:55:10,554 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230324' 2023-03-24 21:55:10,554 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230324 for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:10,555 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230324ea6e832756384abbbdebb994807e4349 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230324ea6e832756384abbbdebb994807e4349 2023-03-24 21:55:10,555 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-24 21:55:10,555 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230324' 2023-03-24 21:55:10,555 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230324 for table='testCompactionWithHFileLink' and column='family1' 2023-03-24 21:55:10,555 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-24 21:55:10,557 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:10,560 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:10,563 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:10,564 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:10,570 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:10,571 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:10,588 INFO [Listener at localhost/39907] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=286 (was 243) - Thread LEAK? -, OpenFileDescriptor=755 (was 608) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=346 (was 369), ProcessCount=169 (was 169), AvailableMemoryMB=7930 (was 8008) 2023-03-24 21:55:10,600 INFO [Listener at localhost/39907] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=286, OpenFileDescriptor=755, MaxFileDescriptor=60000, SystemLoadAverage=346, ProcessCount=169, AvailableMemoryMB=7929 2023-03-24 21:55:10,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$15(3014): Client=jenkins//172.31.2.10 creating {NAME => 'ns'} 2023-03-24 21:55:10,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-03-24 21:55:10,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-24 21:55:10,612 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-24 21:55:10,614 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 7 msec 2023-03-24 21:55:10,620 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family2/823ad28e7cf64530833fd0cce2ea54ce as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/823ad28e7cf64530833fd0cce2ea54ce 2023-03-24 21:55:10,624 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/.tmp/family1/1cb7503f5ddd42e3821147b7fa3c3f60 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/1cb7503f5ddd42e3821147b7fa3c3f60 2023-03-24 21:55:10,624 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 80c7f51f73f3bd5497dc15e88fadefa2/family2 of 80c7f51f73f3bd5497dc15e88fadefa2 into 823ad28e7cf64530833fd0cce2ea54ce(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-24 21:55:10,624 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:55:10,624 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2., storeName=80c7f51f73f3bd5497dc15e88fadefa2/family2, priority=12, startTime=1679694909334; duration=1sec 2023-03-24 21:55:10,625 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-03-24 21:55:10,625 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HRegion(2259): Starting compaction of db6e510d903e066fe70066967a0093e2/family1 in testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:10,625 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-8dbaeb15a2174a16b5567bf58b9bbecd, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-0ade5f95824844f1bdb3bf6657afc66c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-c2c3a5c11515473ba6a2ac8f75a53ddb, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-79acd90194484e429e64a28f2b3d5703, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/4627c5b717b3497a95bbf1567a1e407e_SeqId_34_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp, totalSize=28.9 K 2023-03-24 21:55:10,625 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-8dbaeb15a2174a16b5567bf58b9bbecd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694892686 2023-03-24 21:55:10,626 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-0ade5f95824844f1bdb3bf6657afc66c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694893924 2023-03-24 21:55:10,626 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-c2c3a5c11515473ba6a2ac8f75a53ddb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694894626 2023-03-24 21:55:10,627 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-79acd90194484e429e64a28f2b3d5703, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694896542 2023-03-24 21:55:10,627 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting 4627c5b717b3497a95bbf1567a1e407e_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1679694892686 2023-03-24 21:55:10,628 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 80c7f51f73f3bd5497dc15e88fadefa2/family1 of 80c7f51f73f3bd5497dc15e88fadefa2 into 1cb7503f5ddd42e3821147b7fa3c3f60(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-24 21:55:10,628 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:55:10,628 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2., storeName=80c7f51f73f3bd5497dc15e88fadefa2/family1, priority=12, startTime=1679694909333; duration=1sec 2023-03-24 21:55:10,628 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-24 21:55:10,628 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of dd7d45dee3d2e56a7d8a4ae56820f04d/family1 in testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:10,628 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-d27247c35aab4a5e95522f4b4082ee48, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-3a255a9bd6b34f9e82ea9b25a9f3431b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-d9d7de1995344cbab674aca5eed6eee7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-502a23eefb564b108fd0804dd81ccc60, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/1e78d4ad12754b3a8b2c07974f14d019_SeqId_34_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp, totalSize=28.9 K 2023-03-24 21:55:10,628 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-d27247c35aab4a5e95522f4b4082ee48, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694892685 2023-03-24 21:55:10,629 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-3a255a9bd6b34f9e82ea9b25a9f3431b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694893924 2023-03-24 21:55:10,629 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-d9d7de1995344cbab674aca5eed6eee7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694894626 2023-03-24 21:55:10,630 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-502a23eefb564b108fd0804dd81ccc60, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694896542 2023-03-24 21:55:10,630 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 1e78d4ad12754b3a8b2c07974f14d019_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1679694892685 2023-03-24 21:55:10,639 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] throttle.PressureAwareThroughputController(145): db6e510d903e066fe70066967a0093e2#family1#compaction#115 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-03-24 21:55:10,642 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): dd7d45dee3d2e56a7d8a4ae56820f04d#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-03-24 21:55:10,695 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-cd41d6bc7625452d8cd50d0509b4a453, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-57f179ca64e64316ba7cd9ca99b22557, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-00999c369b14425ea335da2d7d948d08, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-343affb19b444075983c87a317f71c54] to archive 2023-03-24 21:55:10,695 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:10,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-cd41d6bc7625452d8cd50d0509b4a453 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-cd41d6bc7625452d8cd50d0509b4a453 2023-03-24 21:55:10,698 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-57f179ca64e64316ba7cd9ca99b22557 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-57f179ca64e64316ba7cd9ca99b22557 2023-03-24 21:55:10,699 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-00999c369b14425ea335da2d7d948d08 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-00999c369b14425ea335da2d7d948d08 2023-03-24 21:55:10,700 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-343affb19b444075983c87a317f71c54 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family1/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-343affb19b444075983c87a317f71c54 2023-03-24 21:55:10,701 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-b0f3a1484555406892bf1e9f9963292d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-7e5ea937345a494e87b1c26c366f8cd7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-a87ee0a5f4f1401a967dae27a4650f1a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-0beec99d01ea409ab5eb29b007b9a14f] to archive 2023-03-24 21:55:10,702 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:10,703 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-b0f3a1484555406892bf1e9f9963292d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-b0f3a1484555406892bf1e9f9963292d 2023-03-24 21:55:10,703 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-7e5ea937345a494e87b1c26c366f8cd7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-7e5ea937345a494e87b1c26c366f8cd7 2023-03-24 21:55:10,704 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-a87ee0a5f4f1401a967dae27a4650f1a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-a87ee0a5f4f1401a967dae27a4650f1a 2023-03-24 21:55:10,705 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-0beec99d01ea409ab5eb29b007b9a14f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/family2/testCompactionWithHFileLink=80c7f51f73f3bd5497dc15e88fadefa2-0beec99d01ea409ab5eb29b007b9a14f 2023-03-24 21:55:10,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-24 21:55:10,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-24 21:55:10,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-03-24 21:55:10,715 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:55:10,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-03-24 21:55:10,716 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:55:10,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-24 21:55:10,718 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac 2023-03-24 21:55:10,718 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b 2023-03-24 21:55:10,718 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67 2023-03-24 21:55:10,718 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac empty. 2023-03-24 21:55:10,718 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67 empty. 2023-03-24 21:55:10,718 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b empty. 2023-03-24 21:55:10,719 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac 2023-03-24 21:55:10,719 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67 2023-03-24 21:55:10,719 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b 2023-03-24 21:55:10,719 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-03-24 21:55:10,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-24 21:55:11,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-24 21:55:11,125 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-03-24 21:55:11,126 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 5850c24bdc8a7800630587b6a7a58fac, NAME => 'ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:55:11,127 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => a5fe12acb3b1b5a6f77d77af001cfb67, NAME => 'ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:55:11,127 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => 5e6585bbe76c8302dd7ff1c176f5c30b, NAME => 'ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:55:11,132 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:11,132 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing a5fe12acb3b1b5a6f77d77af001cfb67, disabling compactions & flushes 2023-03-24 21:55:11,132 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:11,133 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:11,133 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. after waiting 0 ms 2023-03-24 21:55:11,133 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:11,133 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:11,133 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for a5fe12acb3b1b5a6f77d77af001cfb67: 2023-03-24 21:55:11,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-24 21:55:11,422 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-24 21:55:11,458 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family1/f0699e61ee794fb4a19fb91575d26156 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/f0699e61ee794fb4a19fb91575d26156 2023-03-24 21:55:11,463 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in db6e510d903e066fe70066967a0093e2/family1 of db6e510d903e066fe70066967a0093e2 into f0699e61ee794fb4a19fb91575d26156(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:11,463 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:55:11,463 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1679694910150; duration=0sec 2023-03-24 21:55:11,463 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-24 21:55:11,463 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-24 21:55:11,465 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-24 21:55:11,465 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HStore(1912): dd7d45dee3d2e56a7d8a4ae56820f04d/family2 is initiating minor compaction (all files) 2023-03-24 21:55:11,465 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HRegion(2259): Starting compaction of dd7d45dee3d2e56a7d8a4ae56820f04d/family2 in testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:11,465 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-265c8226a3a1414583f189917e8a53c7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-aad477eb45dc4b8fbb0b9c5cc5ab0982, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-ff2402da2b7244269d37afebd5825ed5, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-73842ebef8cc4e218b9f1e78ba7eead2] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp, totalSize=20.4 K 2023-03-24 21:55:11,466 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-265c8226a3a1414583f189917e8a53c7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694892685 2023-03-24 21:55:11,466 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-aad477eb45dc4b8fbb0b9c5cc5ab0982, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694893924 2023-03-24 21:55:11,467 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-ff2402da2b7244269d37afebd5825ed5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694894626 2023-03-24 21:55:11,467 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-73842ebef8cc4e218b9f1e78ba7eead2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694896542 2023-03-24 21:55:11,478 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] throttle.PressureAwareThroughputController(145): dd7d45dee3d2e56a7d8a4ae56820f04d#family2#compaction#117 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-03-24 21:55:11,501 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-8dbaeb15a2174a16b5567bf58b9bbecd, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-0ade5f95824844f1bdb3bf6657afc66c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-c2c3a5c11515473ba6a2ac8f75a53ddb, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-79acd90194484e429e64a28f2b3d5703, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/4627c5b717b3497a95bbf1567a1e407e_SeqId_34_] to archive 2023-03-24 21:55:11,501 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:11,503 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-8dbaeb15a2174a16b5567bf58b9bbecd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-8dbaeb15a2174a16b5567bf58b9bbecd 2023-03-24 21:55:11,504 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-0ade5f95824844f1bdb3bf6657afc66c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-0ade5f95824844f1bdb3bf6657afc66c 2023-03-24 21:55:11,505 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-c2c3a5c11515473ba6a2ac8f75a53ddb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-c2c3a5c11515473ba6a2ac8f75a53ddb 2023-03-24 21:55:11,505 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-79acd90194484e429e64a28f2b3d5703 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-79acd90194484e429e64a28f2b3d5703 2023-03-24 21:55:11,506 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/4627c5b717b3497a95bbf1567a1e407e_SeqId_34_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family1/4627c5b717b3497a95bbf1567a1e407e_SeqId_34_ 2023-03-24 21:55:11,532 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:11,532 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:11,533 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing 5850c24bdc8a7800630587b6a7a58fac, disabling compactions & flushes 2023-03-24 21:55:11,533 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing 5e6585bbe76c8302dd7ff1c176f5c30b, disabling compactions & flushes 2023-03-24 21:55:11,533 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:11,533 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:11,533 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:11,533 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:11,533 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. after waiting 0 ms 2023-03-24 21:55:11,533 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. after waiting 0 ms 2023-03-24 21:55:11,533 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:11,533 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:11,533 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:11,533 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:11,533 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 5850c24bdc8a7800630587b6a7a58fac: 2023-03-24 21:55:11,533 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for 5e6585bbe76c8302dd7ff1c176f5c30b: 2023-03-24 21:55:11,535 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:55:11,536 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679694911536"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694911536"}]},"ts":"1679694911536"} 2023-03-24 21:55:11,536 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679694911536"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694911536"}]},"ts":"1679694911536"} 2023-03-24 21:55:11,536 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1679694911536"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694911536"}]},"ts":"1679694911536"} 2023-03-24 21:55:11,537 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-24 21:55:11,538 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:55:11,538 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694911538"}]},"ts":"1679694911538"} 2023-03-24 21:55:11,539 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-03-24 21:55:11,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=5850c24bdc8a7800630587b6a7a58fac, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=5e6585bbe76c8302dd7ff1c176f5c30b, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a5fe12acb3b1b5a6f77d77af001cfb67, ASSIGN}] 2023-03-24 21:55:11,544 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=5e6585bbe76c8302dd7ff1c176f5c30b, ASSIGN 2023-03-24 21:55:11,545 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=5850c24bdc8a7800630587b6a7a58fac, ASSIGN 2023-03-24 21:55:11,545 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a5fe12acb3b1b5a6f77d77af001cfb67, ASSIGN 2023-03-24 21:55:11,545 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=5850c24bdc8a7800630587b6a7a58fac, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:55:11,545 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a5fe12acb3b1b5a6f77d77af001cfb67, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:55:11,545 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=5e6585bbe76c8302dd7ff1c176f5c30b, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:55:11,697 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=a5fe12acb3b1b5a6f77d77af001cfb67, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:11,698 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=5850c24bdc8a7800630587b6a7a58fac, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:11,698 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=5e6585bbe76c8302dd7ff1c176f5c30b, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:11,698 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679694911697"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694911697"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694911697"}]},"ts":"1679694911697"} 2023-03-24 21:55:11,698 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1679694911697"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694911697"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694911697"}]},"ts":"1679694911697"} 2023-03-24 21:55:11,698 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679694911697"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694911697"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694911697"}]},"ts":"1679694911697"} 2023-03-24 21:55:11,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=108, state=RUNNABLE; OpenRegionProcedure a5fe12acb3b1b5a6f77d77af001cfb67, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:11,700 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=107, state=RUNNABLE; OpenRegionProcedure 5e6585bbe76c8302dd7ff1c176f5c30b, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:11,700 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=106, state=RUNNABLE; OpenRegionProcedure 5850c24bdc8a7800630587b6a7a58fac, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:11,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-24 21:55:11,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:11,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5e6585bbe76c8302dd7ff1c176f5c30b, NAME => 'ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b.', STARTKEY => '1', ENDKEY => '2'} 2023-03-24 21:55:11,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 5e6585bbe76c8302dd7ff1c176f5c30b 2023-03-24 21:55:11,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:11,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 5e6585bbe76c8302dd7ff1c176f5c30b 2023-03-24 21:55:11,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 5e6585bbe76c8302dd7ff1c176f5c30b 2023-03-24 21:55:11,856 INFO [StoreOpener-5e6585bbe76c8302dd7ff1c176f5c30b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5e6585bbe76c8302dd7ff1c176f5c30b 2023-03-24 21:55:11,857 DEBUG [StoreOpener-5e6585bbe76c8302dd7ff1c176f5c30b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1 2023-03-24 21:55:11,857 DEBUG [StoreOpener-5e6585bbe76c8302dd7ff1c176f5c30b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1 2023-03-24 21:55:11,858 INFO [StoreOpener-5e6585bbe76c8302dd7ff1c176f5c30b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5e6585bbe76c8302dd7ff1c176f5c30b columnFamilyName family1 2023-03-24 21:55:11,859 INFO [StoreOpener-5e6585bbe76c8302dd7ff1c176f5c30b-1] regionserver.HStore(310): Store=5e6585bbe76c8302dd7ff1c176f5c30b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:11,859 INFO [StoreOpener-5e6585bbe76c8302dd7ff1c176f5c30b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 5e6585bbe76c8302dd7ff1c176f5c30b 2023-03-24 21:55:11,859 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family1/1aed70a1165241bc982d5570da0ce670 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/1aed70a1165241bc982d5570da0ce670 2023-03-24 21:55:11,860 DEBUG [StoreOpener-5e6585bbe76c8302dd7ff1c176f5c30b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2 2023-03-24 21:55:11,860 DEBUG [StoreOpener-5e6585bbe76c8302dd7ff1c176f5c30b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2 2023-03-24 21:55:11,861 INFO [StoreOpener-5e6585bbe76c8302dd7ff1c176f5c30b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5e6585bbe76c8302dd7ff1c176f5c30b columnFamilyName family2 2023-03-24 21:55:11,862 INFO [StoreOpener-5e6585bbe76c8302dd7ff1c176f5c30b-1] regionserver.HStore(310): Store=5e6585bbe76c8302dd7ff1c176f5c30b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:11,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b 2023-03-24 21:55:11,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b 2023-03-24 21:55:11,864 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in dd7d45dee3d2e56a7d8a4ae56820f04d/family1 of dd7d45dee3d2e56a7d8a4ae56820f04d into 1aed70a1165241bc982d5570da0ce670(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-24 21:55:11,864 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:55:11,864 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1679694910546; duration=1sec 2023-03-24 21:55:11,864 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-24 21:55:11,864 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-24 21:55:11,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:55:11,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 5e6585bbe76c8302dd7ff1c176f5c30b 2023-03-24 21:55:11,867 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-24 21:55:11,867 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1912): db6e510d903e066fe70066967a0093e2/family2 is initiating minor compaction (all files) 2023-03-24 21:55:11,867 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of db6e510d903e066fe70066967a0093e2/family2 in testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:11,867 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-9430792f9a7c45b2b628cf070eabbb41, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-0eb237e6e4534356b0c3eb93951ca94b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-6b11f68343084dc3bc8a631c212cf15c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-9f76b2a9f280438bb6a623955f71adae] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp, totalSize=20.4 K 2023-03-24 21:55:11,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:55:11,868 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-9430792f9a7c45b2b628cf070eabbb41, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694892686 2023-03-24 21:55:11,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 5e6585bbe76c8302dd7ff1c176f5c30b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59766613, jitterRate=-0.10940806567668915}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:55:11,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 5e6585bbe76c8302dd7ff1c176f5c30b: 2023-03-24 21:55:11,868 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-0eb237e6e4534356b0c3eb93951ca94b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694893924 2023-03-24 21:55:11,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b., pid=110, masterSystemTime=1679694911851 2023-03-24 21:55:11,869 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-6b11f68343084dc3bc8a631c212cf15c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694894626 2023-03-24 21:55:11,870 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-9f76b2a9f280438bb6a623955f71adae, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694896542 2023-03-24 21:55:11,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:11,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:11,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:11,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5850c24bdc8a7800630587b6a7a58fac, NAME => 'ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac.', STARTKEY => '', ENDKEY => '1'} 2023-03-24 21:55:11,871 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=5e6585bbe76c8302dd7ff1c176f5c30b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:11,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 5850c24bdc8a7800630587b6a7a58fac 2023-03-24 21:55:11,871 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1679694911871"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694911871"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694911871"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694911871"}]},"ts":"1679694911871"} 2023-03-24 21:55:11,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:11,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 5850c24bdc8a7800630587b6a7a58fac 2023-03-24 21:55:11,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 5850c24bdc8a7800630587b6a7a58fac 2023-03-24 21:55:11,873 INFO [StoreOpener-5850c24bdc8a7800630587b6a7a58fac-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5850c24bdc8a7800630587b6a7a58fac 2023-03-24 21:55:11,874 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=107 2023-03-24 21:55:11,874 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=107, state=SUCCESS; OpenRegionProcedure 5e6585bbe76c8302dd7ff1c176f5c30b, server=jenkins-hbase9.apache.org,41807,1679694819103 in 173 msec 2023-03-24 21:55:11,875 DEBUG [StoreOpener-5850c24bdc8a7800630587b6a7a58fac-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1 2023-03-24 21:55:11,875 DEBUG [StoreOpener-5850c24bdc8a7800630587b6a7a58fac-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1 2023-03-24 21:55:11,875 INFO [StoreOpener-5850c24bdc8a7800630587b6a7a58fac-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5850c24bdc8a7800630587b6a7a58fac columnFamilyName family1 2023-03-24 21:55:11,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=5e6585bbe76c8302dd7ff1c176f5c30b, ASSIGN in 331 msec 2023-03-24 21:55:11,876 INFO [StoreOpener-5850c24bdc8a7800630587b6a7a58fac-1] regionserver.HStore(310): Store=5850c24bdc8a7800630587b6a7a58fac/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:11,876 INFO [StoreOpener-5850c24bdc8a7800630587b6a7a58fac-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 5850c24bdc8a7800630587b6a7a58fac 2023-03-24 21:55:11,878 DEBUG [StoreOpener-5850c24bdc8a7800630587b6a7a58fac-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2 2023-03-24 21:55:11,878 DEBUG [StoreOpener-5850c24bdc8a7800630587b6a7a58fac-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2 2023-03-24 21:55:11,878 INFO [StoreOpener-5850c24bdc8a7800630587b6a7a58fac-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5850c24bdc8a7800630587b6a7a58fac columnFamilyName family2 2023-03-24 21:55:11,879 INFO [StoreOpener-5850c24bdc8a7800630587b6a7a58fac-1] regionserver.HStore(310): Store=5850c24bdc8a7800630587b6a7a58fac/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:11,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac 2023-03-24 21:55:11,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac 2023-03-24 21:55:11,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:55:11,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 5850c24bdc8a7800630587b6a7a58fac 2023-03-24 21:55:11,884 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): db6e510d903e066fe70066967a0093e2#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-03-24 21:55:11,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:55:11,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 5850c24bdc8a7800630587b6a7a58fac; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72849304, jitterRate=0.08553922176361084}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:55:11,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 5850c24bdc8a7800630587b6a7a58fac: 2023-03-24 21:55:11,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac., pid=111, masterSystemTime=1679694911851 2023-03-24 21:55:11,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:11,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:11,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:11,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a5fe12acb3b1b5a6f77d77af001cfb67, NAME => 'ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67.', STARTKEY => '2', ENDKEY => ''} 2023-03-24 21:55:11,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction a5fe12acb3b1b5a6f77d77af001cfb67 2023-03-24 21:55:11,888 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=5850c24bdc8a7800630587b6a7a58fac, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:11,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:11,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for a5fe12acb3b1b5a6f77d77af001cfb67 2023-03-24 21:55:11,888 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679694911888"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694911888"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694911888"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694911888"}]},"ts":"1679694911888"} 2023-03-24 21:55:11,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for a5fe12acb3b1b5a6f77d77af001cfb67 2023-03-24 21:55:11,890 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=106 2023-03-24 21:55:11,891 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=106, state=SUCCESS; OpenRegionProcedure 5850c24bdc8a7800630587b6a7a58fac, server=jenkins-hbase9.apache.org,41807,1679694819103 in 189 msec 2023-03-24 21:55:11,891 INFO [StoreOpener-a5fe12acb3b1b5a6f77d77af001cfb67-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a5fe12acb3b1b5a6f77d77af001cfb67 2023-03-24 21:55:11,892 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=5850c24bdc8a7800630587b6a7a58fac, ASSIGN in 348 msec 2023-03-24 21:55:11,893 DEBUG [StoreOpener-a5fe12acb3b1b5a6f77d77af001cfb67-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1 2023-03-24 21:55:11,893 DEBUG [StoreOpener-a5fe12acb3b1b5a6f77d77af001cfb67-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1 2023-03-24 21:55:11,893 INFO [StoreOpener-a5fe12acb3b1b5a6f77d77af001cfb67-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5fe12acb3b1b5a6f77d77af001cfb67 columnFamilyName family1 2023-03-24 21:55:11,894 INFO [StoreOpener-a5fe12acb3b1b5a6f77d77af001cfb67-1] regionserver.HStore(310): Store=a5fe12acb3b1b5a6f77d77af001cfb67/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:11,894 INFO [StoreOpener-a5fe12acb3b1b5a6f77d77af001cfb67-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a5fe12acb3b1b5a6f77d77af001cfb67 2023-03-24 21:55:11,895 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-d27247c35aab4a5e95522f4b4082ee48, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-3a255a9bd6b34f9e82ea9b25a9f3431b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-d9d7de1995344cbab674aca5eed6eee7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-502a23eefb564b108fd0804dd81ccc60, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/1e78d4ad12754b3a8b2c07974f14d019_SeqId_34_] to archive 2023-03-24 21:55:11,896 DEBUG [StoreOpener-a5fe12acb3b1b5a6f77d77af001cfb67-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2 2023-03-24 21:55:11,896 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:11,896 DEBUG [StoreOpener-a5fe12acb3b1b5a6f77d77af001cfb67-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2 2023-03-24 21:55:11,896 INFO [StoreOpener-a5fe12acb3b1b5a6f77d77af001cfb67-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5fe12acb3b1b5a6f77d77af001cfb67 columnFamilyName family2 2023-03-24 21:55:11,897 INFO [StoreOpener-a5fe12acb3b1b5a6f77d77af001cfb67-1] regionserver.HStore(310): Store=a5fe12acb3b1b5a6f77d77af001cfb67/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:11,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-d27247c35aab4a5e95522f4b4082ee48 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-d27247c35aab4a5e95522f4b4082ee48 2023-03-24 21:55:11,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67 2023-03-24 21:55:11,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67 2023-03-24 21:55:11,900 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-3a255a9bd6b34f9e82ea9b25a9f3431b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-3a255a9bd6b34f9e82ea9b25a9f3431b 2023-03-24 21:55:11,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:55:11,901 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-d9d7de1995344cbab674aca5eed6eee7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-d9d7de1995344cbab674aca5eed6eee7 2023-03-24 21:55:11,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for a5fe12acb3b1b5a6f77d77af001cfb67 2023-03-24 21:55:11,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-502a23eefb564b108fd0804dd81ccc60 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-502a23eefb564b108fd0804dd81ccc60 2023-03-24 21:55:11,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:55:11,904 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/1e78d4ad12754b3a8b2c07974f14d019_SeqId_34_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family1/1e78d4ad12754b3a8b2c07974f14d019_SeqId_34_ 2023-03-24 21:55:11,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened a5fe12acb3b1b5a6f77d77af001cfb67; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68749254, jitterRate=0.024443715810775757}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:55:11,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for a5fe12acb3b1b5a6f77d77af001cfb67: 2023-03-24 21:55:11,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67., pid=109, masterSystemTime=1679694911851 2023-03-24 21:55:11,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:11,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:11,907 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=a5fe12acb3b1b5a6f77d77af001cfb67, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:11,907 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/.tmp/family2/155e48ae6a574e898f88305783a8322a as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/155e48ae6a574e898f88305783a8322a 2023-03-24 21:55:11,907 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679694911907"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694911907"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694911907"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694911907"}]},"ts":"1679694911907"} 2023-03-24 21:55:11,910 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=108 2023-03-24 21:55:11,910 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=108, state=SUCCESS; OpenRegionProcedure a5fe12acb3b1b5a6f77d77af001cfb67, server=jenkins-hbase9.apache.org,41807,1679694819103 in 209 msec 2023-03-24 21:55:11,911 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in db6e510d903e066fe70066967a0093e2/family2 of db6e510d903e066fe70066967a0093e2 into 155e48ae6a574e898f88305783a8322a(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:11,911 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:55:11,912 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2., storeName=db6e510d903e066fe70066967a0093e2/family2, priority=12, startTime=1679694909500; duration=0sec 2023-03-24 21:55:11,912 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-24 21:55:11,912 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=108, resume processing ppid=105 2023-03-24 21:55:11,912 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-24 21:55:11,912 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a5fe12acb3b1b5a6f77d77af001cfb67, ASSIGN in 367 msec 2023-03-24 21:55:11,912 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-24 21:55:11,912 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-24 21:55:11,912 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. because compaction request was cancelled 2023-03-24 21:55:11,912 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-24 21:55:11,913 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:55:11,913 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-24 21:55:11,913 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694911913"}]},"ts":"1679694911913"} 2023-03-24 21:55:11,913 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-24 21:55:11,913 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. because compaction request was cancelled 2023-03-24 21:55:11,915 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-03-24 21:55:11,917 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:55:11,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2040 sec 2023-03-24 21:55:12,000 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-9430792f9a7c45b2b628cf070eabbb41, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-0eb237e6e4534356b0c3eb93951ca94b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-6b11f68343084dc3bc8a631c212cf15c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-9f76b2a9f280438bb6a623955f71adae] to archive 2023-03-24 21:55:12,001 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:12,002 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-9430792f9a7c45b2b628cf070eabbb41 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-9430792f9a7c45b2b628cf070eabbb41 2023-03-24 21:55:12,004 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-0eb237e6e4534356b0c3eb93951ca94b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-0eb237e6e4534356b0c3eb93951ca94b 2023-03-24 21:55:12,005 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-6b11f68343084dc3bc8a631c212cf15c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-6b11f68343084dc3bc8a631c212cf15c 2023-03-24 21:55:12,006 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-9f76b2a9f280438bb6a623955f71adae to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/family2/testCompactionWithHFileLink=db6e510d903e066fe70066967a0093e2-9f76b2a9f280438bb6a623955f71adae 2023-03-24 21:55:12,703 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/.tmp/family2/e6a853edb1854cfc9a99a8d46e2e9099 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/e6a853edb1854cfc9a99a8d46e2e9099 2023-03-24 21:55:12,706 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in dd7d45dee3d2e56a7d8a4ae56820f04d/family2 of dd7d45dee3d2e56a7d8a4ae56820f04d into e6a853edb1854cfc9a99a8d46e2e9099(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-24 21:55:12,706 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:55:12,706 INFO [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d., storeName=dd7d45dee3d2e56a7d8a4ae56820f04d/family2, priority=12, startTime=1679694909436; duration=1sec 2023-03-24 21:55:12,707 DEBUG [RS:0;jenkins-hbase9:41807-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:55:12,795 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-265c8226a3a1414583f189917e8a53c7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-aad477eb45dc4b8fbb0b9c5cc5ab0982, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-ff2402da2b7244269d37afebd5825ed5, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-73842ebef8cc4e218b9f1e78ba7eead2] to archive 2023-03-24 21:55:12,795 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:12,797 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-265c8226a3a1414583f189917e8a53c7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-265c8226a3a1414583f189917e8a53c7 2023-03-24 21:55:12,798 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-aad477eb45dc4b8fbb0b9c5cc5ab0982 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-aad477eb45dc4b8fbb0b9c5cc5ab0982 2023-03-24 21:55:12,799 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-ff2402da2b7244269d37afebd5825ed5 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-ff2402da2b7244269d37afebd5825ed5 2023-03-24 21:55:12,800 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-73842ebef8cc4e218b9f1e78ba7eead2 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/family2/testCompactionWithHFileLink=dd7d45dee3d2e56a7d8a4ae56820f04d-73842ebef8cc4e218b9f1e78ba7eead2 2023-03-24 21:55:12,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-24 21:55:12,822 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-03-24 21:55:12,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:12,828 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:12,828 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:12,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:12,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-24 21:55:12,831 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-24 21:55:12,831 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:12,831 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-24 21:55:12,831 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-24 21:55:12,832 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:12,832 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:12,834 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:12,834 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:12,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:12,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:12,834 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:12,834 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:12,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:12,834 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:12,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:12,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:12,835 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-24 21:55:12,835 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-24 21:55:12,835 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-24 21:55:12,835 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:12,835 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-24 21:55:12,835 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:55:12,835 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:12,835 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:12,835 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:12,835 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. started... 2023-03-24 21:55:12,835 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:12,835 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. started... 2023-03-24 21:55:12,835 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. started... 2023-03-24 21:55:12,837 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 5850c24bdc8a7800630587b6a7a58fac 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-24 21:55:12,837 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 5e6585bbe76c8302dd7ff1c176f5c30b 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-24 21:55:12,837 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a5fe12acb3b1b5a6f77d77af001cfb67 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-24 21:55:13,249 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324f27d37f047714a27986ff4230e620313 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230324f27d37f047714a27986ff4230e620313 2023-03-24 21:55:13,249 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303243f4bf6cc62414c79a133ce03d457736c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303243f4bf6cc62414c79a133ce03d457736c 2023-03-24 21:55:13,249 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303243da2d59ec77e40e7ba22b96923556422 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303243da2d59ec77e40e7ba22b96923556422 2023-03-24 21:55:13,657 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family1/e752eb58e3994e2bb70614c4c31768ef 2023-03-24 21:55:13,657 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family1/b41682d9f3994c1abaad500d83752008 2023-03-24 21:55:13,657 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family1/7b76f10a8ecc482186f13e27fc1e55be 2023-03-24 21:55:14,071 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324ce0d3e58a7d74e438b48d66ec25bb9aa to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230324ce0d3e58a7d74e438b48d66ec25bb9aa 2023-03-24 21:55:14,071 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032404c987558a9446cea93ce2c66555fe9d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023032404c987558a9446cea93ce2c66555fe9d 2023-03-24 21:55:14,072 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032498aa983c2dc54920bfdeeb6956f3ec0d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023032498aa983c2dc54920bfdeeb6956f3ec0d 2023-03-24 21:55:14,477 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family2/162804a5d8f74c0a9d1242fc8f49e50f 2023-03-24 21:55:14,477 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family2/2878fa1defbe4429824b9ad395e15297 2023-03-24 21:55:14,477 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family2/93865e5a41fb4ae39521d27ad5887d3e 2023-03-24 21:55:14,481 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family1/e752eb58e3994e2bb70614c4c31768ef as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/e752eb58e3994e2bb70614c4c31768ef 2023-03-24 21:55:14,481 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family1/7b76f10a8ecc482186f13e27fc1e55be as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/7b76f10a8ecc482186f13e27fc1e55be 2023-03-24 21:55:14,481 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family1/b41682d9f3994c1abaad500d83752008 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/b41682d9f3994c1abaad500d83752008 2023-03-24 21:55:14,485 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/b41682d9f3994c1abaad500d83752008, entries=4, sequenceid=5, filesize=5.4 K 2023-03-24 21:55:14,485 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/7b76f10a8ecc482186f13e27fc1e55be, entries=4, sequenceid=5, filesize=5.4 K 2023-03-24 21:55:14,485 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/e752eb58e3994e2bb70614c4c31768ef, entries=4, sequenceid=5, filesize=5.4 K 2023-03-24 21:55:14,486 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family2/2878fa1defbe4429824b9ad395e15297 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/2878fa1defbe4429824b9ad395e15297 2023-03-24 21:55:14,486 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family2/162804a5d8f74c0a9d1242fc8f49e50f as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/162804a5d8f74c0a9d1242fc8f49e50f 2023-03-24 21:55:14,486 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family2/93865e5a41fb4ae39521d27ad5887d3e as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/93865e5a41fb4ae39521d27ad5887d3e 2023-03-24 21:55:14,490 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/162804a5d8f74c0a9d1242fc8f49e50f, entries=2, sequenceid=5, filesize=5.1 K 2023-03-24 21:55:14,490 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/2878fa1defbe4429824b9ad395e15297, entries=2, sequenceid=5, filesize=5.1 K 2023-03-24 21:55:14,490 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 5850c24bdc8a7800630587b6a7a58fac in 1653ms, sequenceid=5, compaction requested=false 2023-03-24 21:55:14,490 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 5e6585bbe76c8302dd7ff1c176f5c30b in 1653ms, sequenceid=5, compaction requested=false 2023-03-24 21:55:14,491 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-24 21:55:14,491 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/93865e5a41fb4ae39521d27ad5887d3e, entries=2, sequenceid=5, filesize=5.1 K 2023-03-24 21:55:14,491 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-24 21:55:14,491 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 5850c24bdc8a7800630587b6a7a58fac: 2023-03-24 21:55:14,492 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 5e6585bbe76c8302dd7ff1c176f5c30b: 2023-03-24 21:55:14,492 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:14,492 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a5fe12acb3b1b5a6f77d77af001cfb67 in 1655ms, sequenceid=5, compaction requested=false 2023-03-24 21:55:14,492 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:55:14,492 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a5fe12acb3b1b5a6f77d77af001cfb67: 2023-03-24 21:55:14,492 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:14,492 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:14,492 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:55:14,492 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:55:14,492 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:55:14,492 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,493 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-24 21:55:14,493 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-24 21:55:14,494 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,494 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:14,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:14,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:14,495 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:14,495 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:14,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:14,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:14,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:14,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:14,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-24 21:55:14,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f1d69d1[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:14,496 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-24 21:55:14,496 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:14,497 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:14,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:14,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:14,498 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-24 21:55:14,498 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-24 21:55:14,498 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,498 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:14,498 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:14,500 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-24 21:55:14,500 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:14,500 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-24 21:55:14,500 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:14,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:14,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:14,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:14,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:14,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:14,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:14,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:14,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-24 21:55:14,502 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-24 21:55:14,502 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:14,502 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:14,502 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-24 21:55:14,502 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:14,504 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:14,504 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:14,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:14,504 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:14,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:14,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:14,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:14,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:14,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:14,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:14,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:14,504 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:14,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:14,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:14,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:14,506 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:14,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:14,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:14,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:14,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:14,508 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:14,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:14,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:14,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:14,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-24 21:55:14,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:14,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:14,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:14,508 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:14,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:14,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:14,508 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:14,508 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:14,609 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:14,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:14,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:14,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:14,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:14,613 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:14,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:14,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-24 21:55:14,614 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-24 21:55:14,615 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:14,615 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-24 21:55:14,615 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-24 21:55:14,615 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:14,615 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:14,616 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,616 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:14,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:14,617 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:14,617 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:14,617 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:14,617 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-24 21:55:14,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-24 21:55:14,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-24 21:55:14,618 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-24 21:55:14,618 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:14,618 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-24 21:55:14,618 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:14,618 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. started... 2023-03-24 21:55:14,618 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:14,618 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:55:14,618 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:14,618 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:14,618 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. started... 2023-03-24 21:55:14,618 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. started... 2023-03-24 21:55:14,618 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 5850c24bdc8a7800630587b6a7a58fac 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-24 21:55:14,618 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a5fe12acb3b1b5a6f77d77af001cfb67 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-24 21:55:14,619 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 5e6585bbe76c8302dd7ff1c176f5c30b 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-24 21:55:14,635 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324e0cf8909aed54a5fac0e474f2d0cbe49 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230324e0cf8909aed54a5fac0e474f2d0cbe49 2023-03-24 21:55:15,032 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324a95f11a1ab9e4abfb9ac2d1e691d2b22 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230324a95f11a1ab9e4abfb9ac2d1e691d2b22 2023-03-24 21:55:15,032 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324e7b7e6b6765243e4946e77cae174ea49 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230324e7b7e6b6765243e4946e77cae174ea49 2023-03-24 21:55:15,039 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family1/41868645722a4d608f2d1fa29f1b5980 2023-03-24 21:55:15,437 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family1/a845cb551f5e4e68a8155d095975e025 2023-03-24 21:55:15,437 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family1/488a3a501f4147f2be15de33f18cd149 2023-03-24 21:55:15,457 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032401b54a14b26e401aae08f8a8d2116ae6 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023032401b54a14b26e401aae08f8a8d2116ae6 2023-03-24 21:55:15,458 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303248bba0a138f624edcb45f4ba3d7df34ca to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303248bba0a138f624edcb45f4ba3d7df34ca 2023-03-24 21:55:15,458 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303246b80b969407c4ccc84992a2e437c4b4a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303246b80b969407c4ccc84992a2e437c4b4a 2023-03-24 21:55:15,467 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family2/4237193c68654ec4b29baa1fdbd7252c 2023-03-24 21:55:15,471 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family2/e5ca7a79f8e348eea8b4c5267761251f 2023-03-24 21:55:15,472 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family2/954ae745ced74142808eedfca0d42190 2023-03-24 21:55:15,475 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family1/488a3a501f4147f2be15de33f18cd149 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/488a3a501f4147f2be15de33f18cd149 2023-03-24 21:55:15,476 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family1/41868645722a4d608f2d1fa29f1b5980 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/41868645722a4d608f2d1fa29f1b5980 2023-03-24 21:55:15,480 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/488a3a501f4147f2be15de33f18cd149, entries=4, sequenceid=9, filesize=5.4 K 2023-03-24 21:55:15,480 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family1/a845cb551f5e4e68a8155d095975e025 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/a845cb551f5e4e68a8155d095975e025 2023-03-24 21:55:15,483 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/41868645722a4d608f2d1fa29f1b5980, entries=4, sequenceid=9, filesize=5.4 K 2023-03-24 21:55:15,483 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family2/4237193c68654ec4b29baa1fdbd7252c as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/4237193c68654ec4b29baa1fdbd7252c 2023-03-24 21:55:15,483 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family2/e5ca7a79f8e348eea8b4c5267761251f as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/e5ca7a79f8e348eea8b4c5267761251f 2023-03-24 21:55:15,486 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/a845cb551f5e4e68a8155d095975e025, entries=4, sequenceid=9, filesize=5.4 K 2023-03-24 21:55:15,488 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family2/954ae745ced74142808eedfca0d42190 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/954ae745ced74142808eedfca0d42190 2023-03-24 21:55:15,495 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/954ae745ced74142808eedfca0d42190, entries=2, sequenceid=9, filesize=5.1 K 2023-03-24 21:55:15,496 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 5850c24bdc8a7800630587b6a7a58fac in 878ms, sequenceid=9, compaction requested=false 2023-03-24 21:55:15,496 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/e5ca7a79f8e348eea8b4c5267761251f, entries=2, sequenceid=9, filesize=5.1 K 2023-03-24 21:55:15,496 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 5850c24bdc8a7800630587b6a7a58fac: 2023-03-24 21:55:15,496 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:15,496 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:55:15,497 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 5e6585bbe76c8302dd7ff1c176f5c30b in 879ms, sequenceid=9, compaction requested=false 2023-03-24 21:55:15,497 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 5e6585bbe76c8302dd7ff1c176f5c30b: 2023-03-24 21:55:15,497 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:15,497 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:55:15,502 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/4237193c68654ec4b29baa1fdbd7252c, entries=2, sequenceid=9, filesize=5.1 K 2023-03-24 21:55:15,503 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a5fe12acb3b1b5a6f77d77af001cfb67 in 885ms, sequenceid=9, compaction requested=false 2023-03-24 21:55:15,503 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a5fe12acb3b1b5a6f77d77af001cfb67: 2023-03-24 21:55:15,503 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:15,503 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:55:15,503 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:55:15,503 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,503 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-24 21:55:15,503 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-24 21:55:15,506 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:15,506 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:15,507 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:15,507 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:15,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:15,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:15,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-24 21:55:15,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10a1d245[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:15,508 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-24 21:55:15,508 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:15,511 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:15,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:15,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:15,511 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-24 21:55:15,511 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-24 21:55:15,511 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:15,511 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,512 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:15,516 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,516 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-24 21:55:15,516 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:15,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:15,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:15,516 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-24 21:55:15,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:15,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:15,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:15,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:15,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:15,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:15,521 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-24 21:55:15,521 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-24 21:55:15,522 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:15,522 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:15,522 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-24 21:55:15,522 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:15,523 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:15,523 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:15,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:15,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:15,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:15,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:15,524 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:15,525 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:15,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:15,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:15,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:15,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:15,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:15,526 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:15,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:15,529 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,530 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:15,530 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:15,530 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,530 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:15,530 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:15,530 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:15,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:15,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:15,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-24 21:55:15,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:15,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:15,530 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:15,530 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:15,530 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:15,530 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:15,530 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:15,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:15,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:15,531 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-24 21:55:15,631 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:15,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:15,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:15,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:15,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:15,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:15,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:15,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-24 21:55:15,656 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-24 21:55:15,656 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:15,656 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-24 21:55:15,656 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-24 21:55:15,657 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:15,657 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:15,659 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,659 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:15,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:15,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:15,659 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:15,659 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:15,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:15,660 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:15,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:15,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:15,660 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-24 21:55:15,660 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-24 21:55:15,660 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-24 21:55:15,661 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-24 21:55:15,661 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:15,661 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-24 21:55:15,661 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:15,661 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. started... 2023-03-24 21:55:15,661 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:55:15,661 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:15,661 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:15,661 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a5fe12acb3b1b5a6f77d77af001cfb67 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-24 21:55:15,661 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:15,661 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. started... 2023-03-24 21:55:15,661 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. started... 2023-03-24 21:55:15,661 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 5e6585bbe76c8302dd7ff1c176f5c30b 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-24 21:55:15,661 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 5850c24bdc8a7800630587b6a7a58fac 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-24 21:55:15,696 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032444076a36851b47a7a400512d7e66518e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023032444076a36851b47a7a400512d7e66518e 2023-03-24 21:55:15,700 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303243d202d5ef7e649d28a29bbe04e075e3e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303243d202d5ef7e649d28a29bbe04e075e3e 2023-03-24 21:55:15,704 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324cf6ae84b9f574a80b8ccf8c67811335d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230324cf6ae84b9f574a80b8ccf8c67811335d 2023-03-24 21:55:15,721 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family1/688b80f30c2744549321cd9913fbdac2 2023-03-24 21:55:15,744 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303241e6f7b1b5efd40e8bdf757a7a3473a10 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303241e6f7b1b5efd40e8bdf757a7a3473a10 2023-03-24 21:55:15,751 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family2/d2ac7cec70c84fe8a99fb11120e9d528 2023-03-24 21:55:15,755 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family1/688b80f30c2744549321cd9913fbdac2 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/688b80f30c2744549321cd9913fbdac2 2023-03-24 21:55:15,759 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/688b80f30c2744549321cd9913fbdac2, entries=4, sequenceid=13, filesize=5.4 K 2023-03-24 21:55:15,759 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family2/d2ac7cec70c84fe8a99fb11120e9d528 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/d2ac7cec70c84fe8a99fb11120e9d528 2023-03-24 21:55:15,763 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/d2ac7cec70c84fe8a99fb11120e9d528, entries=2, sequenceid=13, filesize=5.1 K 2023-03-24 21:55:15,764 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 5e6585bbe76c8302dd7ff1c176f5c30b in 103ms, sequenceid=13, compaction requested=true 2023-03-24 21:55:15,764 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 5e6585bbe76c8302dd7ff1c176f5c30b: 2023-03-24 21:55:15,764 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:15,764 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:55:16,117 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family1/1b3f09cf4f8e4440b7384a5926d0a0b7 2023-03-24 21:55:16,121 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family1/0d11426a94a94f2fb264b177e649adde 2023-03-24 21:55:16,536 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032436689e681aea493cadc5b8c8db1c8bbd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023032436689e681aea493cadc5b8c8db1c8bbd 2023-03-24 21:55:16,543 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303245956bbbd8dcd40c2b6a701e8646576fc to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303245956bbbd8dcd40c2b6a701e8646576fc 2023-03-24 21:55:16,941 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family2/e1cf977915324adfb196dca2e94be6a2 2023-03-24 21:55:16,946 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family1/0d11426a94a94f2fb264b177e649adde as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/0d11426a94a94f2fb264b177e649adde 2023-03-24 21:55:16,947 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family2/cd394ebc6c9949b3a5008b6f4cfc0cf1 2023-03-24 21:55:16,950 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/0d11426a94a94f2fb264b177e649adde, entries=4, sequenceid=13, filesize=5.4 K 2023-03-24 21:55:16,951 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family2/e1cf977915324adfb196dca2e94be6a2 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/e1cf977915324adfb196dca2e94be6a2 2023-03-24 21:55:16,951 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family1/1b3f09cf4f8e4440b7384a5926d0a0b7 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/1b3f09cf4f8e4440b7384a5926d0a0b7 2023-03-24 21:55:16,955 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/e1cf977915324adfb196dca2e94be6a2, entries=2, sequenceid=13, filesize=5.1 K 2023-03-24 21:55:16,955 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/1b3f09cf4f8e4440b7384a5926d0a0b7, entries=4, sequenceid=13, filesize=5.4 K 2023-03-24 21:55:16,956 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a5fe12acb3b1b5a6f77d77af001cfb67 in 1295ms, sequenceid=13, compaction requested=true 2023-03-24 21:55:16,956 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a5fe12acb3b1b5a6f77d77af001cfb67: 2023-03-24 21:55:16,956 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:16,956 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family2/cd394ebc6c9949b3a5008b6f4cfc0cf1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/cd394ebc6c9949b3a5008b6f4cfc0cf1 2023-03-24 21:55:16,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:55:16,960 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/cd394ebc6c9949b3a5008b6f4cfc0cf1, entries=2, sequenceid=13, filesize=5.1 K 2023-03-24 21:55:16,961 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 5850c24bdc8a7800630587b6a7a58fac in 1300ms, sequenceid=13, compaction requested=true 2023-03-24 21:55:16,961 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 5850c24bdc8a7800630587b6a7a58fac: 2023-03-24 21:55:16,961 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:16,961 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:55:16,961 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:55:16,961 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,961 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-24 21:55:16,961 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-24 21:55:16,963 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:16,963 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:16,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:16,963 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:16,963 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:16,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:16,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:16,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:16,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:16,965 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-24 21:55:16,965 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56b7b682[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:16,965 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-24 21:55:16,965 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:16,966 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:16,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:16,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:16,966 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-24 21:55:16,966 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-24 21:55:16,966 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,967 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:16,966 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:16,968 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,968 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-24 21:55:16,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:16,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:16,968 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:16,968 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-24 21:55:16,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:16,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:16,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:16,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:16,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:16,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:16,971 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-24 21:55:16,971 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-24 21:55:16,971 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:16,971 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:16,971 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-24 21:55:16,971 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:16,974 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:16,974 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:16,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:16,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:16,974 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:16,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:16,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:16,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:16,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:16,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:16,974 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:16,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:16,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:16,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:16,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:16,976 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:16,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:16,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,979 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,979 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:16,979 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:16,979 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:16,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-24 21:55:16,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:16,979 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:16,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:16,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-24 21:55:16,979 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:16,980 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:16,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:16,979 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:16,980 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:16,980 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:16,980 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:16,980 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:16,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:16,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:17,080 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:17,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:17,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:17,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:17,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:17,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:17,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:17,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-24 21:55:17,087 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-24 21:55:17,087 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:17,087 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-24 21:55:17,087 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-24 21:55:17,088 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:17,088 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:17,089 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:17,089 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:17,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:17,089 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,089 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:17,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:17,090 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:17,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:17,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:17,090 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-24 21:55:17,090 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-24 21:55:17,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-24 21:55:17,090 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-24 21:55:17,090 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:17,090 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-24 21:55:17,091 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:17,091 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. started... 2023-03-24 21:55:17,091 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:55:17,091 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a5fe12acb3b1b5a6f77d77af001cfb67 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-24 21:55:17,091 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:17,091 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:17,091 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. started... 2023-03-24 21:55:17,091 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:17,091 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. started... 2023-03-24 21:55:17,091 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 5850c24bdc8a7800630587b6a7a58fac 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-24 21:55:17,091 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 5e6585bbe76c8302dd7ff1c176f5c30b 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-24 21:55:17,122 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303245ca59b0ab5e64158b2f71b166c92fcfe to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303245ca59b0ab5e64158b2f71b166c92fcfe 2023-03-24 21:55:17,124 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303243181cd4481834a7c9e25df12cac5af3a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303243181cd4481834a7c9e25df12cac5af3a 2023-03-24 21:55:17,133 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family1/32e66ce115bb4c29919c9c8ab0b4ce1e 2023-03-24 21:55:17,356 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-24 21:55:17,521 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032427ae8f4a0a824f29bbb842a1f0fbb12f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023032427ae8f4a0a824f29bbb842a1f0fbb12f 2023-03-24 21:55:17,530 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family1/cb240f6a645845549ee1d1a7928ab8a7 2023-03-24 21:55:17,533 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family1/d8c97bc56fa14b7a9087866186136dfb 2023-03-24 21:55:17,550 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032409644a5811834b93b58123bd7c504a89 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023032409644a5811834b93b58123bd7c504a89 2023-03-24 21:55:17,566 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family2/ce9ab252700e419ab4cf33188e35e1c5 2023-03-24 21:55:17,567 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032428c463fcc5fd469bb4f34b7fefa0cfb1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023032428c463fcc5fd469bb4f34b7fefa0cfb1 2023-03-24 21:55:17,570 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324b5c9b322fc91467e89cc68407fb8e598 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230324b5c9b322fc91467e89cc68407fb8e598 2023-03-24 21:55:17,570 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family1/32e66ce115bb4c29919c9c8ab0b4ce1e as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/32e66ce115bb4c29919c9c8ab0b4ce1e 2023-03-24 21:55:17,575 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/32e66ce115bb4c29919c9c8ab0b4ce1e, entries=4, sequenceid=17, filesize=5.4 K 2023-03-24 21:55:17,575 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family2/862f8ae745b546b882e81b5cf78cf7ba 2023-03-24 21:55:17,576 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family2/ce9ab252700e419ab4cf33188e35e1c5 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/ce9ab252700e419ab4cf33188e35e1c5 2023-03-24 21:55:17,580 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family1/cb240f6a645845549ee1d1a7928ab8a7 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/cb240f6a645845549ee1d1a7928ab8a7 2023-03-24 21:55:17,580 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family2/9012a567fff042d9bba2eab0e4c82818 2023-03-24 21:55:17,581 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/ce9ab252700e419ab4cf33188e35e1c5, entries=2, sequenceid=17, filesize=5.1 K 2023-03-24 21:55:17,582 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 5e6585bbe76c8302dd7ff1c176f5c30b in 491ms, sequenceid=17, compaction requested=true 2023-03-24 21:55:17,582 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 5e6585bbe76c8302dd7ff1c176f5c30b: 2023-03-24 21:55:17,582 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:17,582 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:55:17,584 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family1/d8c97bc56fa14b7a9087866186136dfb as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/d8c97bc56fa14b7a9087866186136dfb 2023-03-24 21:55:17,584 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/cb240f6a645845549ee1d1a7928ab8a7, entries=4, sequenceid=17, filesize=5.4 K 2023-03-24 21:55:17,585 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family2/862f8ae745b546b882e81b5cf78cf7ba as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/862f8ae745b546b882e81b5cf78cf7ba 2023-03-24 21:55:17,589 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/d8c97bc56fa14b7a9087866186136dfb, entries=4, sequenceid=17, filesize=5.4 K 2023-03-24 21:55:17,589 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/862f8ae745b546b882e81b5cf78cf7ba, entries=2, sequenceid=17, filesize=5.1 K 2023-03-24 21:55:17,589 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family2/9012a567fff042d9bba2eab0e4c82818 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/9012a567fff042d9bba2eab0e4c82818 2023-03-24 21:55:17,590 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 5850c24bdc8a7800630587b6a7a58fac in 499ms, sequenceid=17, compaction requested=true 2023-03-24 21:55:17,590 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 5850c24bdc8a7800630587b6a7a58fac: 2023-03-24 21:55:17,590 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:17,590 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:55:17,593 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/9012a567fff042d9bba2eab0e4c82818, entries=2, sequenceid=17, filesize=5.1 K 2023-03-24 21:55:17,593 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a5fe12acb3b1b5a6f77d77af001cfb67 in 502ms, sequenceid=17, compaction requested=true 2023-03-24 21:55:17,594 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a5fe12acb3b1b5a6f77d77af001cfb67: 2023-03-24 21:55:17,594 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:17,594 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:55:17,594 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:55:17,594 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,594 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-24 21:55:17,594 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-24 21:55:17,597 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,597 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:17,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:17,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:17,597 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:17,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:17,597 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:17,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:17,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:17,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:17,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-24 21:55:17,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ae4d854[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:17,599 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-24 21:55:17,599 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:17,601 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:17,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:17,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:17,601 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-24 21:55:17,601 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-24 21:55:17,602 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:17,602 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,602 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:17,604 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-24 21:55:17,604 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:17,604 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,604 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-24 21:55:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:17,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:17,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:17,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:17,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:17,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-24 21:55:17,606 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-24 21:55:17,606 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:17,606 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:17,606 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-24 21:55:17,606 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:17,608 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:17,608 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:17,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:17,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:17,609 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:17,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:17,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:17,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:17,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:17,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:17,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:17,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:17,612 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:17,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:17,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:17,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,613 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:17,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:17,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,616 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:17,616 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:17,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:17,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:17,617 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,617 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:17,617 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:17,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-24 21:55:17,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:17,617 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:17,617 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:17,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:17,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:17,617 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:17,618 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,618 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:17,618 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:17,618 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:17,718 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:17,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:17,731 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:17,742 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:17,754 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:17,755 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:17,792 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:17,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:17,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-24 21:55:17,793 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-24 21:55:17,794 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:17,794 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-24 21:55:17,794 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-24 21:55:17,794 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:17,794 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:17,795 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,795 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:17,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:17,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:17,795 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:17,795 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:17,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:17,796 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:17,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:17,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:17,796 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-24 21:55:17,796 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-24 21:55:17,797 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-24 21:55:17,797 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-24 21:55:17,797 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:17,797 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-24 21:55:17,797 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:17,797 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. started... 2023-03-24 21:55:17,797 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a5fe12acb3b1b5a6f77d77af001cfb67 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-24 21:55:17,798 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:17,798 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. started... 2023-03-24 21:55:17,798 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:55:17,798 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:17,798 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 5e6585bbe76c8302dd7ff1c176f5c30b 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-24 21:55:17,798 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:17,798 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. started... 2023-03-24 21:55:17,798 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 5850c24bdc8a7800630587b6a7a58fac 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-24 21:55:17,842 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family1/ff06f79545ad4d88a9e364e61bb2ee73 2023-03-24 21:55:17,845 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff06f79545ad4d88a9e364e61bb2ee73 2023-03-24 21:55:18,242 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family1/66011a38e6e24d3c86c1fb2d5186a98b 2023-03-24 21:55:18,245 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 66011a38e6e24d3c86c1fb2d5186a98b 2023-03-24 21:55:18,260 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family2/66943e4dc7dd400bba17cde17d30527a 2023-03-24 21:55:18,264 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 66943e4dc7dd400bba17cde17d30527a 2023-03-24 21:55:18,265 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family1/ff06f79545ad4d88a9e364e61bb2ee73 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/ff06f79545ad4d88a9e364e61bb2ee73 2023-03-24 21:55:18,269 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff06f79545ad4d88a9e364e61bb2ee73 2023-03-24 21:55:18,269 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/ff06f79545ad4d88a9e364e61bb2ee73, entries=3, sequenceid=23, filesize=5.0 K 2023-03-24 21:55:18,270 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family2/66943e4dc7dd400bba17cde17d30527a as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/66943e4dc7dd400bba17cde17d30527a 2023-03-24 21:55:18,276 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 66943e4dc7dd400bba17cde17d30527a 2023-03-24 21:55:18,276 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/66943e4dc7dd400bba17cde17d30527a, entries=1, sequenceid=23, filesize=5.0 K 2023-03-24 21:55:18,277 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 5e6585bbe76c8302dd7ff1c176f5c30b in 479ms, sequenceid=23, compaction requested=true 2023-03-24 21:55:18,277 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 5e6585bbe76c8302dd7ff1c176f5c30b: 2023-03-24 21:55:18,277 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:18,277 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:55:18,631 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family1/eb1877e0bd3a4d529b73b240525c8b15 2023-03-24 21:55:18,635 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb1877e0bd3a4d529b73b240525c8b15 2023-03-24 21:55:18,658 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family2/ef1b8845323b49a0850b5fe2857de9d4 2023-03-24 21:55:18,663 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ef1b8845323b49a0850b5fe2857de9d4 2023-03-24 21:55:18,664 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family1/66011a38e6e24d3c86c1fb2d5186a98b as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/66011a38e6e24d3c86c1fb2d5186a98b 2023-03-24 21:55:18,669 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 66011a38e6e24d3c86c1fb2d5186a98b 2023-03-24 21:55:18,669 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/66011a38e6e24d3c86c1fb2d5186a98b, entries=3, sequenceid=23, filesize=5.0 K 2023-03-24 21:55:18,670 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family2/ef1b8845323b49a0850b5fe2857de9d4 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/ef1b8845323b49a0850b5fe2857de9d4 2023-03-24 21:55:18,674 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ef1b8845323b49a0850b5fe2857de9d4 2023-03-24 21:55:18,674 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/ef1b8845323b49a0850b5fe2857de9d4, entries=1, sequenceid=23, filesize=5.0 K 2023-03-24 21:55:18,675 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 5850c24bdc8a7800630587b6a7a58fac in 877ms, sequenceid=23, compaction requested=true 2023-03-24 21:55:18,675 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 5850c24bdc8a7800630587b6a7a58fac: 2023-03-24 21:55:18,675 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:18,675 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:55:19,049 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family2/a317374315da4264a4af703bfc196883 2023-03-24 21:55:19,053 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a317374315da4264a4af703bfc196883 2023-03-24 21:55:19,054 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family1/eb1877e0bd3a4d529b73b240525c8b15 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/eb1877e0bd3a4d529b73b240525c8b15 2023-03-24 21:55:19,058 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb1877e0bd3a4d529b73b240525c8b15 2023-03-24 21:55:19,059 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/eb1877e0bd3a4d529b73b240525c8b15, entries=3, sequenceid=23, filesize=5.0 K 2023-03-24 21:55:19,059 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family2/a317374315da4264a4af703bfc196883 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/a317374315da4264a4af703bfc196883 2023-03-24 21:55:19,063 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a317374315da4264a4af703bfc196883 2023-03-24 21:55:19,063 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/a317374315da4264a4af703bfc196883, entries=1, sequenceid=23, filesize=5.0 K 2023-03-24 21:55:19,064 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a5fe12acb3b1b5a6f77d77af001cfb67 in 1267ms, sequenceid=23, compaction requested=true 2023-03-24 21:55:19,064 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a5fe12acb3b1b5a6f77d77af001cfb67: 2023-03-24 21:55:19,064 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:19,064 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:55:19,064 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:55:19,064 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,064 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-24 21:55:19,064 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-24 21:55:19,066 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,066 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:19,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:19,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:19,066 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:19,066 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:19,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:19,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:19,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:19,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:19,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-24 21:55:19,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ed37338[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:19,068 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-24 21:55:19,068 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:19,070 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:19,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:19,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:19,071 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-24 21:55:19,071 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-24 21:55:19,071 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:19,071 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,071 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:19,075 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,075 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-24 21:55:19,075 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:19,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,075 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-24 21:55:19,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:19,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:19,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:19,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:19,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:19,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:19,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:19,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:19,078 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-24 21:55:19,078 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-24 21:55:19,078 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:19,078 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:19,078 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-24 21:55:19,078 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:19,079 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:19,079 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:19,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:19,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:19,079 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:19,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:19,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:19,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:19,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:19,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:19,080 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:19,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:19,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:19,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:19,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:19,083 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:19,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-24 21:55:19,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,086 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,086 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:19,086 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:19,086 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:19,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-24 21:55:19,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:19,086 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-24 21:55:19,087 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:19,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:19,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:19,087 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:19,086 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:19,087 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:19,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-24 21:55:19,087 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:19,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:19,087 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-24 21:55:19,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:19,087 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-24 21:55:19,187 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:19,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:19,188 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:19,189 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): a5fe12acb3b1b5a6f77d77af001cfb67/family1 is initiating major compaction (all files) 2023-03-24 21:55:19,189 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:19,189 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:19,189 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of a5fe12acb3b1b5a6f77d77af001cfb67/family1 in ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:19,189 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/e752eb58e3994e2bb70614c4c31768ef, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/488a3a501f4147f2be15de33f18cd149, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/0d11426a94a94f2fb264b177e649adde, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/d8c97bc56fa14b7a9087866186136dfb, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/eb1877e0bd3a4d529b73b240525c8b15] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp, totalSize=26.6 K 2023-03-24 21:55:19,190 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting e752eb58e3994e2bb70614c4c31768ef, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694912828 2023-03-24 21:55:19,190 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 488a3a501f4147f2be15de33f18cd149, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694914611 2023-03-24 21:55:19,191 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 0d11426a94a94f2fb264b177e649adde, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694915633 2023-03-24 21:55:19,191 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting d8c97bc56fa14b7a9087866186136dfb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694917082 2023-03-24 21:55:19,192 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting eb1877e0bd3a4d529b73b240525c8b15, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-24 21:55:19,232 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): a5fe12acb3b1b5a6f77d77af001cfb67#family1#compaction#149 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:19,257 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023032424febfdbc28444bcbbe091c869ecb495_del 2023-03-24 21:55:19,258 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032424febfdbc28444bcbbe091c869ecb495_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023032424febfdbc28444bcbbe091c869ecb495_del 2023-03-24 21:55:19,288 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 07f97e4c594448a5966f1ba49a1e679a 2023-03-24 21:55:19,289 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family1/07f97e4c594448a5966f1ba49a1e679a as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/07f97e4c594448a5966f1ba49a1e679a 2023-03-24 21:55:19,292 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 07f97e4c594448a5966f1ba49a1e679a 2023-03-24 21:55:19,293 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a5fe12acb3b1b5a6f77d77af001cfb67/family1 of a5fe12acb3b1b5a6f77d77af001cfb67 into 07f97e4c594448a5966f1ba49a1e679a(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:19,293 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for a5fe12acb3b1b5a6f77d77af001cfb67: 2023-03-24 21:55:19,293 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:19,293 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): a5fe12acb3b1b5a6f77d77af001cfb67/family2 is initiating major compaction (all files) 2023-03-24 21:55:19,293 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:19,294 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:19,294 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of a5fe12acb3b1b5a6f77d77af001cfb67/family2 in ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:19,295 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/93865e5a41fb4ae39521d27ad5887d3e, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/4237193c68654ec4b29baa1fdbd7252c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/e1cf977915324adfb196dca2e94be6a2, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/9012a567fff042d9bba2eab0e4c82818, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/a317374315da4264a4af703bfc196883] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp, totalSize=25.3 K 2023-03-24 21:55:19,295 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/e752eb58e3994e2bb70614c4c31768ef, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/488a3a501f4147f2be15de33f18cd149, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/0d11426a94a94f2fb264b177e649adde, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/d8c97bc56fa14b7a9087866186136dfb, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/eb1877e0bd3a4d529b73b240525c8b15] to archive 2023-03-24 21:55:19,296 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 93865e5a41fb4ae39521d27ad5887d3e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694912828 2023-03-24 21:55:19,296 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:19,296 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 4237193c68654ec4b29baa1fdbd7252c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694914611 2023-03-24 21:55:19,297 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting e1cf977915324adfb196dca2e94be6a2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694915633 2023-03-24 21:55:19,297 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 9012a567fff042d9bba2eab0e4c82818, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694917082 2023-03-24 21:55:19,298 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting a317374315da4264a4af703bfc196883, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-24 21:55:19,298 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/e752eb58e3994e2bb70614c4c31768ef to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/e752eb58e3994e2bb70614c4c31768ef 2023-03-24 21:55:19,301 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/488a3a501f4147f2be15de33f18cd149 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/488a3a501f4147f2be15de33f18cd149 2023-03-24 21:55:19,302 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/0d11426a94a94f2fb264b177e649adde to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/0d11426a94a94f2fb264b177e649adde 2023-03-24 21:55:19,303 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/d8c97bc56fa14b7a9087866186136dfb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/d8c97bc56fa14b7a9087866186136dfb 2023-03-24 21:55:19,305 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/eb1877e0bd3a4d529b73b240525c8b15 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/eb1877e0bd3a4d529b73b240525c8b15 2023-03-24 21:55:19,311 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): a5fe12acb3b1b5a6f77d77af001cfb67#family2#compaction#150 average throughput is 0.20 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:19,731 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230324d1644a8c72204036bacd8b9999ace696_del 2023-03-24 21:55:19,731 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324d1644a8c72204036bacd8b9999ace696_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230324d1644a8c72204036bacd8b9999ace696_del 2023-03-24 21:55:20,545 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1ad16726afef49cd981d619a76ff37f9 2023-03-24 21:55:20,545 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/.tmp/family2/1ad16726afef49cd981d619a76ff37f9 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/1ad16726afef49cd981d619a76ff37f9 2023-03-24 21:55:20,549 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1ad16726afef49cd981d619a76ff37f9 2023-03-24 21:55:20,550 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a5fe12acb3b1b5a6f77d77af001cfb67/family2 of a5fe12acb3b1b5a6f77d77af001cfb67 into 1ad16726afef49cd981d619a76ff37f9(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-24 21:55:20,550 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for a5fe12acb3b1b5a6f77d77af001cfb67: 2023-03-24 21:55:20,550 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:20,550 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): 5e6585bbe76c8302dd7ff1c176f5c30b/family1 is initiating major compaction (all files) 2023-03-24 21:55:20,550 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:20,550 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:20,550 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of 5e6585bbe76c8302dd7ff1c176f5c30b/family1 in ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:20,551 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/b41682d9f3994c1abaad500d83752008, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/41868645722a4d608f2d1fa29f1b5980, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/688b80f30c2744549321cd9913fbdac2, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/32e66ce115bb4c29919c9c8ab0b4ce1e, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/ff06f79545ad4d88a9e364e61bb2ee73] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp, totalSize=26.6 K 2023-03-24 21:55:20,551 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting b41682d9f3994c1abaad500d83752008, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694912828 2023-03-24 21:55:20,551 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 41868645722a4d608f2d1fa29f1b5980, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694914611 2023-03-24 21:55:20,552 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 688b80f30c2744549321cd9913fbdac2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694915633 2023-03-24 21:55:20,552 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 32e66ce115bb4c29919c9c8ab0b4ce1e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694917082 2023-03-24 21:55:20,553 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting ff06f79545ad4d88a9e364e61bb2ee73, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-24 21:55:20,561 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): 5e6585bbe76c8302dd7ff1c176f5c30b#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:20,568 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230324b7b3a5ca952e4f46a69d8adabea47e45_del 2023-03-24 21:55:20,568 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324b7b3a5ca952e4f46a69d8adabea47e45_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230324b7b3a5ca952e4f46a69d8adabea47e45_del 2023-03-24 21:55:20,594 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/93865e5a41fb4ae39521d27ad5887d3e, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/4237193c68654ec4b29baa1fdbd7252c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/e1cf977915324adfb196dca2e94be6a2, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/9012a567fff042d9bba2eab0e4c82818, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/a317374315da4264a4af703bfc196883] to archive 2023-03-24 21:55:20,595 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:20,597 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/93865e5a41fb4ae39521d27ad5887d3e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/93865e5a41fb4ae39521d27ad5887d3e 2023-03-24 21:55:20,598 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/4237193c68654ec4b29baa1fdbd7252c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/4237193c68654ec4b29baa1fdbd7252c 2023-03-24 21:55:20,599 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/e1cf977915324adfb196dca2e94be6a2 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/e1cf977915324adfb196dca2e94be6a2 2023-03-24 21:55:20,600 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/9012a567fff042d9bba2eab0e4c82818 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/9012a567fff042d9bba2eab0e4c82818 2023-03-24 21:55:20,601 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/a317374315da4264a4af703bfc196883 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family2/a317374315da4264a4af703bfc196883 2023-03-24 21:55:20,981 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7432dcf2c5fa494b9c60ed3e73c0552c 2023-03-24 21:55:20,981 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family1/7432dcf2c5fa494b9c60ed3e73c0552c as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/7432dcf2c5fa494b9c60ed3e73c0552c 2023-03-24 21:55:20,985 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7432dcf2c5fa494b9c60ed3e73c0552c 2023-03-24 21:55:20,986 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 5e6585bbe76c8302dd7ff1c176f5c30b/family1 of 5e6585bbe76c8302dd7ff1c176f5c30b into 7432dcf2c5fa494b9c60ed3e73c0552c(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:20,986 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for 5e6585bbe76c8302dd7ff1c176f5c30b: 2023-03-24 21:55:20,987 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:20,987 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): 5e6585bbe76c8302dd7ff1c176f5c30b/family2 is initiating major compaction (all files) 2023-03-24 21:55:20,987 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:20,987 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:20,987 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of 5e6585bbe76c8302dd7ff1c176f5c30b/family2 in ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:20,987 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/2878fa1defbe4429824b9ad395e15297, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/e5ca7a79f8e348eea8b4c5267761251f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/d2ac7cec70c84fe8a99fb11120e9d528, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/ce9ab252700e419ab4cf33188e35e1c5, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/66943e4dc7dd400bba17cde17d30527a] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp, totalSize=25.3 K 2023-03-24 21:55:20,988 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 2878fa1defbe4429824b9ad395e15297, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694912828 2023-03-24 21:55:20,988 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting e5ca7a79f8e348eea8b4c5267761251f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694914611 2023-03-24 21:55:20,988 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting d2ac7cec70c84fe8a99fb11120e9d528, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694915633 2023-03-24 21:55:20,989 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting ce9ab252700e419ab4cf33188e35e1c5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694917082 2023-03-24 21:55:20,989 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 66943e4dc7dd400bba17cde17d30527a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-24 21:55:20,995 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/b41682d9f3994c1abaad500d83752008, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/41868645722a4d608f2d1fa29f1b5980, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/688b80f30c2744549321cd9913fbdac2, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/32e66ce115bb4c29919c9c8ab0b4ce1e, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/ff06f79545ad4d88a9e364e61bb2ee73] to archive 2023-03-24 21:55:20,996 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:20,999 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/b41682d9f3994c1abaad500d83752008 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/b41682d9f3994c1abaad500d83752008 2023-03-24 21:55:20,999 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): 5e6585bbe76c8302dd7ff1c176f5c30b#family2#compaction#152 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:21,001 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/41868645722a4d608f2d1fa29f1b5980 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/41868645722a4d608f2d1fa29f1b5980 2023-03-24 21:55:21,003 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/688b80f30c2744549321cd9913fbdac2 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/688b80f30c2744549321cd9913fbdac2 2023-03-24 21:55:21,004 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/32e66ce115bb4c29919c9c8ab0b4ce1e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/32e66ce115bb4c29919c9c8ab0b4ce1e 2023-03-24 21:55:21,005 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/ff06f79545ad4d88a9e364e61bb2ee73 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/ff06f79545ad4d88a9e364e61bb2ee73 2023-03-24 21:55:21,008 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230324a9694f5654f5461b8db9fcc93a58c5ad_del 2023-03-24 21:55:21,008 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324a9694f5654f5461b8db9fcc93a58c5ad_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230324a9694f5654f5461b8db9fcc93a58c5ad_del 2023-03-24 21:55:21,034 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a445a444c6674408aa5ee3106afaaa5b 2023-03-24 21:55:21,034 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/.tmp/family2/a445a444c6674408aa5ee3106afaaa5b as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/a445a444c6674408aa5ee3106afaaa5b 2023-03-24 21:55:21,040 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a445a444c6674408aa5ee3106afaaa5b 2023-03-24 21:55:21,041 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 5e6585bbe76c8302dd7ff1c176f5c30b/family2 of 5e6585bbe76c8302dd7ff1c176f5c30b into a445a444c6674408aa5ee3106afaaa5b(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:21,041 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for 5e6585bbe76c8302dd7ff1c176f5c30b: 2023-03-24 21:55:21,041 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:21,041 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): 5850c24bdc8a7800630587b6a7a58fac/family1 is initiating major compaction (all files) 2023-03-24 21:55:21,041 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:21,042 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:21,042 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of 5850c24bdc8a7800630587b6a7a58fac/family1 in ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:21,042 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/7b76f10a8ecc482186f13e27fc1e55be, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/a845cb551f5e4e68a8155d095975e025, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/1b3f09cf4f8e4440b7384a5926d0a0b7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/cb240f6a645845549ee1d1a7928ab8a7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/66011a38e6e24d3c86c1fb2d5186a98b] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp, totalSize=26.6 K 2023-03-24 21:55:21,042 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 7b76f10a8ecc482186f13e27fc1e55be, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694912827 2023-03-24 21:55:21,043 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting a845cb551f5e4e68a8155d095975e025, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694914610 2023-03-24 21:55:21,043 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 1b3f09cf4f8e4440b7384a5926d0a0b7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694915633 2023-03-24 21:55:21,044 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting cb240f6a645845549ee1d1a7928ab8a7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694917082 2023-03-24 21:55:21,044 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 66011a38e6e24d3c86c1fb2d5186a98b, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-24 21:55:21,062 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): 5850c24bdc8a7800630587b6a7a58fac#family1#compaction#153 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:21,094 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/2878fa1defbe4429824b9ad395e15297, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/e5ca7a79f8e348eea8b4c5267761251f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/d2ac7cec70c84fe8a99fb11120e9d528, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/ce9ab252700e419ab4cf33188e35e1c5, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/66943e4dc7dd400bba17cde17d30527a] to archive 2023-03-24 21:55:21,095 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:21,097 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/2878fa1defbe4429824b9ad395e15297 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/2878fa1defbe4429824b9ad395e15297 2023-03-24 21:55:21,098 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/e5ca7a79f8e348eea8b4c5267761251f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/e5ca7a79f8e348eea8b4c5267761251f 2023-03-24 21:55:21,099 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/d2ac7cec70c84fe8a99fb11120e9d528 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/d2ac7cec70c84fe8a99fb11120e9d528 2023-03-24 21:55:21,100 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/ce9ab252700e419ab4cf33188e35e1c5 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/ce9ab252700e419ab4cf33188e35e1c5 2023-03-24 21:55:21,101 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/66943e4dc7dd400bba17cde17d30527a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family2/66943e4dc7dd400bba17cde17d30527a 2023-03-24 21:55:21,482 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230324966bbe2217c24571a0df2849b81f67d2_del 2023-03-24 21:55:21,482 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324966bbe2217c24571a0df2849b81f67d2_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230324966bbe2217c24571a0df2849b81f67d2_del 2023-03-24 21:55:22,298 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ecd052866db140e0bbd32082dfb3b8c5 2023-03-24 21:55:22,298 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family1/ecd052866db140e0bbd32082dfb3b8c5 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/ecd052866db140e0bbd32082dfb3b8c5 2023-03-24 21:55:22,302 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ecd052866db140e0bbd32082dfb3b8c5 2023-03-24 21:55:22,303 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 5850c24bdc8a7800630587b6a7a58fac/family1 of 5850c24bdc8a7800630587b6a7a58fac into ecd052866db140e0bbd32082dfb3b8c5(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-24 21:55:22,303 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for 5850c24bdc8a7800630587b6a7a58fac: 2023-03-24 21:55:22,303 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:22,303 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): 5850c24bdc8a7800630587b6a7a58fac/family2 is initiating major compaction (all files) 2023-03-24 21:55:22,304 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:22,304 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:22,306 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of 5850c24bdc8a7800630587b6a7a58fac/family2 in ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:22,306 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/162804a5d8f74c0a9d1242fc8f49e50f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/954ae745ced74142808eedfca0d42190, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/cd394ebc6c9949b3a5008b6f4cfc0cf1, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/862f8ae745b546b882e81b5cf78cf7ba, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/ef1b8845323b49a0850b5fe2857de9d4] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp, totalSize=25.3 K 2023-03-24 21:55:22,306 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 162804a5d8f74c0a9d1242fc8f49e50f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694912827 2023-03-24 21:55:22,307 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 954ae745ced74142808eedfca0d42190, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694914610 2023-03-24 21:55:22,307 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting cd394ebc6c9949b3a5008b6f4cfc0cf1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694915633 2023-03-24 21:55:22,308 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 862f8ae745b546b882e81b5cf78cf7ba, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694917082 2023-03-24 21:55:22,308 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting ef1b8845323b49a0850b5fe2857de9d4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-24 21:55:22,324 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): 5850c24bdc8a7800630587b6a7a58fac#family2#compaction#154 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:22,340 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230324938d078caecf4f41b282a3a7db37fa3c_del 2023-03-24 21:55:22,341 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324938d078caecf4f41b282a3a7db37fa3c_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230324938d078caecf4f41b282a3a7db37fa3c_del 2023-03-24 21:55:22,397 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/7b76f10a8ecc482186f13e27fc1e55be, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/a845cb551f5e4e68a8155d095975e025, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/1b3f09cf4f8e4440b7384a5926d0a0b7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/cb240f6a645845549ee1d1a7928ab8a7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/66011a38e6e24d3c86c1fb2d5186a98b] to archive 2023-03-24 21:55:22,398 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:22,399 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/7b76f10a8ecc482186f13e27fc1e55be to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/7b76f10a8ecc482186f13e27fc1e55be 2023-03-24 21:55:22,400 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/a845cb551f5e4e68a8155d095975e025 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/a845cb551f5e4e68a8155d095975e025 2023-03-24 21:55:22,402 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/1b3f09cf4f8e4440b7384a5926d0a0b7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/1b3f09cf4f8e4440b7384a5926d0a0b7 2023-03-24 21:55:22,403 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/cb240f6a645845549ee1d1a7928ab8a7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/cb240f6a645845549ee1d1a7928ab8a7 2023-03-24 21:55:22,404 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/66011a38e6e24d3c86c1fb2d5186a98b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/66011a38e6e24d3c86c1fb2d5186a98b 2023-03-24 21:55:23,158 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9309db7f5e7a4abcb588a9e102120399 2023-03-24 21:55:23,159 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/.tmp/family2/9309db7f5e7a4abcb588a9e102120399 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/9309db7f5e7a4abcb588a9e102120399 2023-03-24 21:55:23,162 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9309db7f5e7a4abcb588a9e102120399 2023-03-24 21:55:23,163 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 5850c24bdc8a7800630587b6a7a58fac/family2 of 5850c24bdc8a7800630587b6a7a58fac into 9309db7f5e7a4abcb588a9e102120399(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:23,163 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for 5850c24bdc8a7800630587b6a7a58fac: 2023-03-24 21:55:23,169 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:23,172 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:23,175 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:23,176 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:23,183 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-24 21:55:23,202 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/162804a5d8f74c0a9d1242fc8f49e50f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/954ae745ced74142808eedfca0d42190, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/cd394ebc6c9949b3a5008b6f4cfc0cf1, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/862f8ae745b546b882e81b5cf78cf7ba, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/ef1b8845323b49a0850b5fe2857de9d4] to archive 2023-03-24 21:55:23,203 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:23,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/162804a5d8f74c0a9d1242fc8f49e50f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/162804a5d8f74c0a9d1242fc8f49e50f 2023-03-24 21:55:23,207 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/954ae745ced74142808eedfca0d42190 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/954ae745ced74142808eedfca0d42190 2023-03-24 21:55:23,209 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/cd394ebc6c9949b3a5008b6f4cfc0cf1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/cd394ebc6c9949b3a5008b6f4cfc0cf1 2023-03-24 21:55:23,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/862f8ae745b546b882e81b5cf78cf7ba to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/862f8ae745b546b882e81b5cf78cf7ba 2023-03-24 21:55:23,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/ef1b8845323b49a0850b5fe2857de9d4 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family2/ef1b8845323b49a0850b5fe2857de9d4 2023-03-24 21:55:23,218 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-03-24 21:55:23,222 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230324966bbe2217c24571a0df2849b81f67d2_del 2023-03-24 21:55:23,227 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230324b7b3a5ca952e4f46a69d8adabea47e45_del 2023-03-24 21:55:23,230 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023032424febfdbc28444bcbbe091c869ecb495_del 2023-03-24 21:55:23,230 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-03-24 21:55:23,231 DEBUG [Listener at localhost/39907] zookeeper.ReadOnlyZKClient(139): Connect 0x27ca7234 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:55:23,242 DEBUG [Listener at localhost/39907] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@44a1355, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:55:23,244 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230324 for table='ns:testMinorCompaction' and column='family1' 2023-03-24 21:55:23,244 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230324 for table='ns:testMinorCompaction' and column='family1' 2023-03-24 21:55:23,245 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230324 for table='ns:testMinorCompaction' and column='family1' 2023-03-24 21:55:23,290 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324fe684fcdad8b45ae970bc30de6fb6a0c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230324fe684fcdad8b45ae970bc30de6fb6a0c 2023-03-24 21:55:23,291 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230324 table='ns:testMinorCompaction' column='family1' 2023-03-24 21:55:23,305 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:55:23,307 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:60108, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:55:23,315 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:55:23,318 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:56326, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:55:23,334 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230324/family1/2816978f89bf4d85b8801dd1ee3843d1 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-03-24 21:55:23,335 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230324/family1/2816978f89bf4d85b8801dd1ee3843d1}] 2023-03-24 21:55:23,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230324/family1/2816978f89bf4d85b8801dd1ee3843d1 for inclusion in 5e6585bbe76c8302dd7ff1c176f5c30b/family1 2023-03-24 21:55:23,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-03-24 21:55:23,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-24 21:55:23,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 5e6585bbe76c8302dd7ff1c176f5c30b: 2023-03-24 21:55:23,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230324/family1/2816978f89bf4d85b8801dd1ee3843d1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__b4vol26cet72h7llf7s4cuhjh3p4q2rs5i948qi9rc0mo2342t0f9b0bdg1hepcq/family1/2816978f89bf4d85b8801dd1ee3843d1 2023-03-24 21:55:23,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__b4vol26cet72h7llf7s4cuhjh3p4q2rs5i948qi9rc0mo2342t0f9b0bdg1hepcq/family1/2816978f89bf4d85b8801dd1ee3843d1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/518f1b39905b44d489f0505bb86f309b_SeqId_28_ 2023-03-24 21:55:23,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__b4vol26cet72h7llf7s4cuhjh3p4q2rs5i948qi9rc0mo2342t0f9b0bdg1hepcq/family1/2816978f89bf4d85b8801dd1ee3843d1 into 5e6585bbe76c8302dd7ff1c176f5c30b/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/518f1b39905b44d489f0505bb86f309b_SeqId_28_ - updating store file list. 2023-03-24 21:55:23,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/518f1b39905b44d489f0505bb86f309b_SeqId_28_ into 5e6585bbe76c8302dd7ff1c176f5c30b/family1 2023-03-24 21:55:23,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__b4vol26cet72h7llf7s4cuhjh3p4q2rs5i948qi9rc0mo2342t0f9b0bdg1hepcq/family1/2816978f89bf4d85b8801dd1ee3843d1 into 5e6585bbe76c8302dd7ff1c176f5c30b/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/family1/518f1b39905b44d489f0505bb86f309b_SeqId_28_) 2023-03-24 21:55:23,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__b4vol26cet72h7llf7s4cuhjh3p4q2rs5i948qi9rc0mo2342t0f9b0bdg1hepcq/family1/2816978f89bf4d85b8801dd1ee3843d1 2023-03-24 21:55:23,355 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230324 table='ns:testMinorCompaction' column='family1' 2023-03-24 21:55:23,356 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:23,358 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303243f4bf6cc62414c79a133ce03d457736c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303243f4bf6cc62414c79a133ce03d457736c 2023-03-24 21:55:23,358 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-24 21:55:23,358 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230324' 2023-03-24 21:55:23,358 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230324 for table='ns:testMinorCompaction' and column='family1' 2023-03-24 21:55:23,685 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324bbe3a58ceef246e3b218a666c435b259 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230324bbe3a58ceef246e3b218a666c435b259 2023-03-24 21:55:23,685 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230324 table='ns:testMinorCompaction' column='family1' 2023-03-24 21:55:23,696 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324b2bcaf25083641fcb27b56c71ed3a9aa to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230324b2bcaf25083641fcb27b56c71ed3a9aa 2023-03-24 21:55:23,697 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230324 table='ns:testMinorCompaction' column='family1' 2023-03-24 21:55:23,724 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230324/family1/65bfc6d097544aedbac96dcfd32d16ec first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-03-24 21:55:23,724 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230324/family1/65bfc6d097544aedbac96dcfd32d16ec}] 2023-03-24 21:55:23,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230324/family1/65bfc6d097544aedbac96dcfd32d16ec for inclusion in a5fe12acb3b1b5a6f77d77af001cfb67/family1 2023-03-24 21:55:23,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-03-24 21:55:23,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-24 21:55:23,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for a5fe12acb3b1b5a6f77d77af001cfb67: 2023-03-24 21:55:23,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230324/family1/65bfc6d097544aedbac96dcfd32d16ec to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__q6scs4s9g5gveeftkhqs70h5i112tk01b2sr7ggib7o71uffoh4oqhc5jm7gisfe/family1/65bfc6d097544aedbac96dcfd32d16ec 2023-03-24 21:55:23,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__q6scs4s9g5gveeftkhqs70h5i112tk01b2sr7ggib7o71uffoh4oqhc5jm7gisfe/family1/65bfc6d097544aedbac96dcfd32d16ec as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/96a72a9f221c47d9b59104023b3d2e45_SeqId_28_ 2023-03-24 21:55:23,734 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__q6scs4s9g5gveeftkhqs70h5i112tk01b2sr7ggib7o71uffoh4oqhc5jm7gisfe/family1/65bfc6d097544aedbac96dcfd32d16ec into a5fe12acb3b1b5a6f77d77af001cfb67/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/96a72a9f221c47d9b59104023b3d2e45_SeqId_28_ - updating store file list. 2023-03-24 21:55:23,736 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/96a72a9f221c47d9b59104023b3d2e45_SeqId_28_ into a5fe12acb3b1b5a6f77d77af001cfb67/family1 2023-03-24 21:55:23,736 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__q6scs4s9g5gveeftkhqs70h5i112tk01b2sr7ggib7o71uffoh4oqhc5jm7gisfe/family1/65bfc6d097544aedbac96dcfd32d16ec into a5fe12acb3b1b5a6f77d77af001cfb67/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/family1/96a72a9f221c47d9b59104023b3d2e45_SeqId_28_) 2023-03-24 21:55:23,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__q6scs4s9g5gveeftkhqs70h5i112tk01b2sr7ggib7o71uffoh4oqhc5jm7gisfe/family1/65bfc6d097544aedbac96dcfd32d16ec 2023-03-24 21:55:23,738 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230324 table='ns:testMinorCompaction' column='family1' 2023-03-24 21:55:23,740 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:23,742 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230324/family1/085e2ad7796e4529ad8f08955b582fa6 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-03-24 21:55:23,743 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230324f27d37f047714a27986ff4230e620313 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230324f27d37f047714a27986ff4230e620313 2023-03-24 21:55:23,743 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230324/family1/085e2ad7796e4529ad8f08955b582fa6}] 2023-03-24 21:55:23,743 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-24 21:55:23,743 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230324' 2023-03-24 21:55:23,743 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230324 for table='ns:testMinorCompaction' and column='family1' 2023-03-24 21:55:23,744 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230324/family1/085e2ad7796e4529ad8f08955b582fa6 for inclusion in 5850c24bdc8a7800630587b6a7a58fac/family1 2023-03-24 21:55:23,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-03-24 21:55:23,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last=1 2023-03-24 21:55:23,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 5850c24bdc8a7800630587b6a7a58fac: 2023-03-24 21:55:23,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230324/family1/085e2ad7796e4529ad8f08955b582fa6 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__t8vn9s1n9imd73oedd2ufsfgcn02lbqit6vj2ft1vhg8bfve2irpht59d57ughtg/family1/085e2ad7796e4529ad8f08955b582fa6 2023-03-24 21:55:23,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__t8vn9s1n9imd73oedd2ufsfgcn02lbqit6vj2ft1vhg8bfve2irpht59d57ughtg/family1/085e2ad7796e4529ad8f08955b582fa6 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/15137410dba94cf6985d9bb01e94c881_SeqId_28_ 2023-03-24 21:55:23,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__t8vn9s1n9imd73oedd2ufsfgcn02lbqit6vj2ft1vhg8bfve2irpht59d57ughtg/family1/085e2ad7796e4529ad8f08955b582fa6 into 5850c24bdc8a7800630587b6a7a58fac/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/15137410dba94cf6985d9bb01e94c881_SeqId_28_ - updating store file list. 2023-03-24 21:55:23,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/15137410dba94cf6985d9bb01e94c881_SeqId_28_ into 5850c24bdc8a7800630587b6a7a58fac/family1 2023-03-24 21:55:23,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__t8vn9s1n9imd73oedd2ufsfgcn02lbqit6vj2ft1vhg8bfve2irpht59d57ughtg/family1/085e2ad7796e4529ad8f08955b582fa6 into 5850c24bdc8a7800630587b6a7a58fac/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/family1/15137410dba94cf6985d9bb01e94c881_SeqId_28_) 2023-03-24 21:55:23,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__ns_testMinorCompaction__t8vn9s1n9imd73oedd2ufsfgcn02lbqit6vj2ft1vhg8bfve2irpht59d57ughtg/family1/085e2ad7796e4529ad8f08955b582fa6 2023-03-24 21:55:23,754 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230324 table='ns:testMinorCompaction' column='family1' 2023-03-24 21:55:23,755 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:23,756 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303243da2d59ec77e40e7ba22b96923556422 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303243da2d59ec77e40e7ba22b96923556422 2023-03-24 21:55:23,756 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-24 21:55:23,756 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230324' 2023-03-24 21:55:23,757 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230324 for table='ns:testMinorCompaction' and column='family1' 2023-03-24 21:55:23,757 INFO [Listener at localhost/39907] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-03-24 21:55:23,759 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:23,761 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:23,763 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:23,764 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:23,788 INFO [Listener at localhost/39907] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=253 (was 286), OpenFileDescriptor=657 (was 755), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=338 (was 346), ProcessCount=169 (was 169), AvailableMemoryMB=7911 (was 7929) 2023-03-24 21:55:23,802 INFO [Listener at localhost/39907] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=253, OpenFileDescriptor=657, MaxFileDescriptor=60000, SystemLoadAverage=338, ProcessCount=169, AvailableMemoryMB=7910 2023-03-24 21:55:23,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-24 21:55:23,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:23,808 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:55:23,808 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-03-24 21:55:23,809 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:55:23,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-24 21:55:23,810 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a 2023-03-24 21:55:23,810 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a empty. 2023-03-24 21:55:23,811 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a 2023-03-24 21:55:23,811 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-03-24 21:55:23,827 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-24 21:55:23,828 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2b21dad992cbb40d94473c32040a2d9a, NAME => 'testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:55:23,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-24 21:55:24,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-24 21:55:24,243 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:24,244 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 2b21dad992cbb40d94473c32040a2d9a, disabling compactions & flushes 2023-03-24 21:55:24,244 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:24,244 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:24,244 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. after waiting 0 ms 2023-03-24 21:55:24,244 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:24,244 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:24,244 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:24,246 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:55:24,247 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679694924247"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694924247"}]},"ts":"1679694924247"} 2023-03-24 21:55:24,249 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-24 21:55:24,250 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:55:24,250 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694924250"}]},"ts":"1679694924250"} 2023-03-24 21:55:24,251 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-24 21:55:24,253 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2b21dad992cbb40d94473c32040a2d9a, ASSIGN}] 2023-03-24 21:55:24,255 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2b21dad992cbb40d94473c32040a2d9a, ASSIGN 2023-03-24 21:55:24,256 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2b21dad992cbb40d94473c32040a2d9a, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:55:24,407 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2b21dad992cbb40d94473c32040a2d9a, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:24,407 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679694924407"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694924407"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694924407"}]},"ts":"1679694924407"} 2023-03-24 21:55:24,409 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 2b21dad992cbb40d94473c32040a2d9a, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:24,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-24 21:55:24,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:24,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2b21dad992cbb40d94473c32040a2d9a, NAME => 'testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a.', STARTKEY => '', ENDKEY => ''} 2023-03-24 21:55:24,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 2b21dad992cbb40d94473c32040a2d9a 2023-03-24 21:55:24,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:24,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 2b21dad992cbb40d94473c32040a2d9a 2023-03-24 21:55:24,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 2b21dad992cbb40d94473c32040a2d9a 2023-03-24 21:55:24,564 INFO [StoreOpener-2b21dad992cbb40d94473c32040a2d9a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2b21dad992cbb40d94473c32040a2d9a 2023-03-24 21:55:24,566 DEBUG [StoreOpener-2b21dad992cbb40d94473c32040a2d9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1 2023-03-24 21:55:24,566 DEBUG [StoreOpener-2b21dad992cbb40d94473c32040a2d9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1 2023-03-24 21:55:24,566 INFO [StoreOpener-2b21dad992cbb40d94473c32040a2d9a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2b21dad992cbb40d94473c32040a2d9a columnFamilyName family1 2023-03-24 21:55:24,567 INFO [StoreOpener-2b21dad992cbb40d94473c32040a2d9a-1] regionserver.HStore(310): Store=2b21dad992cbb40d94473c32040a2d9a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:24,568 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a 2023-03-24 21:55:24,568 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a 2023-03-24 21:55:24,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 2b21dad992cbb40d94473c32040a2d9a 2023-03-24 21:55:24,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:55:24,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 2b21dad992cbb40d94473c32040a2d9a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68305504, jitterRate=0.01783132553100586}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-24 21:55:24,573 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:24,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a., pid=114, masterSystemTime=1679694924560 2023-03-24 21:55:24,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:24,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:24,575 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2b21dad992cbb40d94473c32040a2d9a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:24,575 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679694924575"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694924575"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694924575"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694924575"}]},"ts":"1679694924575"} 2023-03-24 21:55:24,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-03-24 21:55:24,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 2b21dad992cbb40d94473c32040a2d9a, server=jenkins-hbase9.apache.org,41807,1679694819103 in 168 msec 2023-03-24 21:55:24,579 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-03-24 21:55:24,579 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2b21dad992cbb40d94473c32040a2d9a, ASSIGN in 325 msec 2023-03-24 21:55:24,580 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:55:24,580 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694924580"}]},"ts":"1679694924580"} 2023-03-24 21:55:24,581 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-24 21:55:24,583 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:55:24,584 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 778 msec 2023-03-24 21:55:24,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-24 21:55:24,914 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-03-24 21:55:24,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:24,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:24,950 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:24,950 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:24,950 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:55:24,950 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:55:24,952 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:24,952 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:24,953 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:24,953 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:24,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:24,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:24,954 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:24,954 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:24,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:24,954 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:24,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:24,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:24,955 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:24,955 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:24,955 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:55:24,955 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:24,955 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:55:24,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:55:24,956 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:24,956 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:24,956 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. started... 2023-03-24 21:55:24,957 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b21dad992cbb40d94473c32040a2d9a 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-24 21:55:25,389 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113006edd780d1494652b78259edd9897a3c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113006edd780d1494652b78259edd9897a3c 2023-03-24 21:55:25,793 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/79c64a3156324c919a4da2158e4a5156 2023-03-24 21:55:25,797 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/79c64a3156324c919a4da2158e4a5156 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/79c64a3156324c919a4da2158e4a5156 2023-03-24 21:55:25,801 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/79c64a3156324c919a4da2158e4a5156, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-24 21:55:25,802 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 2b21dad992cbb40d94473c32040a2d9a in 845ms, sequenceid=5, compaction requested=false 2023-03-24 21:55:25,802 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:25,803 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:25,803 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:25,803 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:55:25,803 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:55:25,803 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,803 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:55:25,803 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:55:25,805 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,805 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:25,806 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,806 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:25,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:25,807 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:55:25,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30feaf5d[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:25,808 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:55:25,808 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,809 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,809 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:55:25,809 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:55:25,809 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:25,809 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,809 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:25,810 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,810 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:55:25,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:25,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:25,810 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:25,810 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:55:25,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:25,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:25,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:25,813 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:55:25,813 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:55:25,813 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:25,813 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:25,813 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,813 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:25,815 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,815 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,815 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:25,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:25,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:25,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:25,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:25,815 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:25,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:25,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,816 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:25,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,819 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:25,819 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,819 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:25,819 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:25,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:55:25,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:25,819 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:25,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:25,819 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:25,819 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,819 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,819 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,819 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,820 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:25,820 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:25,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:25,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:25,920 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:25,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:25,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:25,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:25,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,926 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:25,926 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:25,926 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:55:25,926 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:55:25,926 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,927 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,929 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,929 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:25,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:25,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:25,930 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:25,930 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:25,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,930 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:25,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,930 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,931 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:55:25,931 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:25,931 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:55:25,931 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:25,931 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:55:25,931 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:55:25,931 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:25,931 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:25,931 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. started... 2023-03-24 21:55:25,931 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b21dad992cbb40d94473c32040a2d9a 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-24 21:55:26,344 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128fe687c9daba24876b43ef9b41dc4f060 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128fe687c9daba24876b43ef9b41dc4f060 2023-03-24 21:55:26,750 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/646897bb59ec4b7f8712ede9d306ca12 2023-03-24 21:55:26,755 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/646897bb59ec4b7f8712ede9d306ca12 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/646897bb59ec4b7f8712ede9d306ca12 2023-03-24 21:55:26,759 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/646897bb59ec4b7f8712ede9d306ca12, entries=1, sequenceid=9, filesize=5.0 K 2023-03-24 21:55:26,760 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 2b21dad992cbb40d94473c32040a2d9a in 829ms, sequenceid=9, compaction requested=false 2023-03-24 21:55:26,760 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:26,760 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:26,760 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:55:26,760 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:55:26,760 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,761 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:55:26,761 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:55:26,763 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,763 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:26,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:26,763 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,763 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:26,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:26,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:55:26,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b961c66[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:26,765 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:55:26,765 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,766 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,766 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:55:26,766 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:55:26,766 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:26,766 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,766 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:26,767 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,768 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:55:26,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:26,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:26,768 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:26,768 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:55:26,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:26,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:26,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:55:26,771 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:55:26,771 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:26,771 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:26,771 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,771 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:26,772 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,772 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:26,772 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:26,773 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:26,774 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:26,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,777 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,777 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:26,777 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:26,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:26,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:26,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:26,777 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:26,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:26,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:26,778 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:26,777 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,778 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,778 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,778 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,778 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,778 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:26,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:55:26,778 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:26,879 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:26,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:26,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:26,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:26,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,885 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:26,885 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:26,885 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:55:26,885 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:55:26,886 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,886 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,890 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:26,890 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:26,891 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,891 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,891 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:26,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,892 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,892 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:55:26,892 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,892 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:55:26,892 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:26,892 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:55:26,892 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:55:26,892 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:26,892 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:26,892 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. started... 2023-03-24 21:55:26,893 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b21dad992cbb40d94473c32040a2d9a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:55:26,904 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201071a0628cce74485aa872039228cd891 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201071a0628cce74485aa872039228cd891 2023-03-24 21:55:26,909 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/143547e18f1f44e78635d377894b5f46 2023-03-24 21:55:26,914 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/143547e18f1f44e78635d377894b5f46 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/143547e18f1f44e78635d377894b5f46 2023-03-24 21:55:26,918 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/143547e18f1f44e78635d377894b5f46, entries=1, sequenceid=13, filesize=4.9 K 2023-03-24 21:55:26,919 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b21dad992cbb40d94473c32040a2d9a in 26ms, sequenceid=13, compaction requested=true 2023-03-24 21:55:26,919 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:26,920 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:26,920 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:55:26,920 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:55:26,920 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,920 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:55:26,920 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:55:26,921 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,921 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:26,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:26,922 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,922 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:26,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:26,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:26,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:26,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:55:26,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@375c5938[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:26,923 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:55:26,923 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,924 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,924 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:55:26,924 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:55:26,924 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:26,924 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,925 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:26,926 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:55:26,926 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,926 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:26,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,926 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:55:26,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:26,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:26,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:26,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:26,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:26,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:26,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:55:26,928 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:55:26,929 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:26,929 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:26,929 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,929 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:26,935 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,935 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,935 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:26,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:26,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:26,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:26,936 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:26,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:26,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:26,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,937 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:26,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,940 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,940 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,940 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:26,940 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:26,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:26,940 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:26,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:26,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:26,940 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:26,940 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:26,940 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,940 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,940 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:26,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:26,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:26,940 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:26,941 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:27,041 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:27,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:27,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:27,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:27,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,047 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:27,047 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:27,048 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:55:27,048 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:55:27,048 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,048 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,050 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,050 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:27,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:27,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:27,050 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,050 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:27,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:27,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,051 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:55:27,051 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,051 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:55:27,052 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:27,052 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:55:27,052 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:55:27,052 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:27,052 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:27,052 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. started... 2023-03-24 21:55:27,052 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b21dad992cbb40d94473c32040a2d9a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:55:27,470 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f3475b2ed08f49f78ba6e77e0305451f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205f3475b2ed08f49f78ba6e77e0305451f 2023-03-24 21:55:27,877 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/0c2f43e5d2e846cf9bd6e3ea21bc730b 2023-03-24 21:55:27,882 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/0c2f43e5d2e846cf9bd6e3ea21bc730b as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/0c2f43e5d2e846cf9bd6e3ea21bc730b 2023-03-24 21:55:27,886 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/0c2f43e5d2e846cf9bd6e3ea21bc730b, entries=1, sequenceid=17, filesize=4.9 K 2023-03-24 21:55:27,887 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b21dad992cbb40d94473c32040a2d9a in 835ms, sequenceid=17, compaction requested=true 2023-03-24 21:55:27,887 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:27,887 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:27,887 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:55:27,888 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:55:27,888 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,888 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:55:27,888 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:55:27,889 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,889 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:27,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:27,890 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,890 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:27,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:27,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:27,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:27,892 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:55:27,892 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@223a658[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:27,892 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:55:27,892 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,895 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,895 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:55:27,895 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:55:27,895 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:27,896 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,896 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:27,897 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:55:27,897 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:27,897 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:55:27,897 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:27,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:27,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:27,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:27,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:27,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:27,909 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:55:27,909 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:55:27,909 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:27,909 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:27,909 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,909 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:27,913 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,913 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:27,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:27,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:27,914 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:27,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:27,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:27,915 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:27,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,917 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:27,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,921 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,921 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:27,921 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:27,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:27,921 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:27,921 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,922 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:27,922 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,922 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,922 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:27,922 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:27,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:27,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:27,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:27,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:27,923 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:27,923 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:28,023 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:28,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:28,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:28,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:28,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,030 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:28,030 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:28,030 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:55:28,030 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:55:28,030 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,030 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,032 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:28,032 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:28,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:28,032 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,032 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:28,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:28,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:55:28,033 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,034 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:55:28,034 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:28,034 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:55:28,034 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:55:28,034 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:28,034 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:28,034 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. started... 2023-03-24 21:55:28,034 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b21dad992cbb40d94473c32040a2d9a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:55:28,051 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228fa7748b7992447838107aab21a9dc051 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228fa7748b7992447838107aab21a9dc051 2023-03-24 21:55:28,077 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/c9f13fa77349465faea350d294067b64 2023-03-24 21:55:28,082 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/c9f13fa77349465faea350d294067b64 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/c9f13fa77349465faea350d294067b64 2023-03-24 21:55:28,088 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/c9f13fa77349465faea350d294067b64, entries=1, sequenceid=21, filesize=4.9 K 2023-03-24 21:55:28,089 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b21dad992cbb40d94473c32040a2d9a in 55ms, sequenceid=21, compaction requested=true 2023-03-24 21:55:28,089 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:28,089 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:28,090 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:55:28,090 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:55:28,090 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,090 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:55:28,090 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:55:28,091 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,092 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:28,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:28,092 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,092 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:28,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:28,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:28,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:28,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:55:28,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47e26506[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:28,094 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:55:28,094 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,095 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,095 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:55:28,095 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:55:28,095 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,095 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:28,095 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:28,097 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,097 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:55:28,097 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,097 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:55:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:28,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:28,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:28,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:28,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:28,100 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:55:28,100 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:55:28,100 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:28,100 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:28,100 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,100 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:28,103 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,103 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:28,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:28,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:28,104 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:28,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:28,105 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:28,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,106 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,111 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,111 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,111 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:28,111 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:28,111 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:28,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:28,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:28,111 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:28,112 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,112 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,112 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,112 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,112 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:28,112 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:28,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:28,212 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:28,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:28,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:28,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:28,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,219 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:28,219 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:28,219 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:55:28,219 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:55:28,219 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,219 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,221 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,221 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:28,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:28,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:28,222 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:28,222 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:28,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,222 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:28,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,222 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:55:28,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:28,224 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:55:28,224 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:28,224 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:55:28,227 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:55:28,227 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:28,227 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:28,227 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. started... 2023-03-24 21:55:28,227 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b21dad992cbb40d94473c32040a2d9a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:55:28,650 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123124bdb7cba8964770af77d2fc217cb1f4 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123124bdb7cba8964770af77d2fc217cb1f4 2023-03-24 21:55:29,055 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/a4409e238bbb4bb686ff1e14eefb0e56 2023-03-24 21:55:29,060 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/a4409e238bbb4bb686ff1e14eefb0e56 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/a4409e238bbb4bb686ff1e14eefb0e56 2023-03-24 21:55:29,063 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/a4409e238bbb4bb686ff1e14eefb0e56, entries=1, sequenceid=25, filesize=4.9 K 2023-03-24 21:55:29,064 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b21dad992cbb40d94473c32040a2d9a in 837ms, sequenceid=25, compaction requested=true 2023-03-24 21:55:29,064 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:29,064 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:29,065 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:55:29,065 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:55:29,065 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,065 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:55:29,065 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:55:29,067 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,067 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:29,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:29,068 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,068 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:29,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:29,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:29,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:29,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:55:29,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2cfc34b0[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:29,069 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:55:29,069 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,071 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,071 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:55:29,071 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:55:29,071 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:29,071 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,071 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:29,072 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:55:29,072 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:29,072 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,072 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:55:29,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:29,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:29,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:29,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:55:29,075 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:55:29,075 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:29,075 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:29,075 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,075 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:29,077 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,077 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:29,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,077 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:29,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:29,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:29,078 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:29,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:29,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:29,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,079 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:29,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,081 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,081 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:29,081 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,081 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:29,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:29,081 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:29,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:29,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:55:29,081 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:29,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:29,082 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:29,082 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:29,082 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,082 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,082 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:29,082 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:29,182 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:29,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:29,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:29,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:29,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,189 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:29,189 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:29,189 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:55:29,189 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:55:29,189 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,189 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,190 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:29,190 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:29,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:29,191 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,191 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:29,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,191 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:29,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,192 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,192 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:55:29,192 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,192 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:55:29,192 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:29,192 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:55:29,192 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:55:29,192 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:29,193 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:29,193 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. started... 2023-03-24 21:55:29,193 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b21dad992cbb40d94473c32040a2d9a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:55:29,607 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010179c6b23c36364179bc619bb0056b1d6f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010179c6b23c36364179bc619bb0056b1d6f 2023-03-24 21:55:29,620 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/3f36365f683d43d0a7c74bffdb79aeda 2023-03-24 21:55:29,625 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/3f36365f683d43d0a7c74bffdb79aeda as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3f36365f683d43d0a7c74bffdb79aeda 2023-03-24 21:55:29,629 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3f36365f683d43d0a7c74bffdb79aeda, entries=1, sequenceid=29, filesize=4.9 K 2023-03-24 21:55:29,630 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b21dad992cbb40d94473c32040a2d9a in 437ms, sequenceid=29, compaction requested=true 2023-03-24 21:55:29,630 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:29,630 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:29,630 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:55:29,630 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:55:29,630 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,630 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:55:29,630 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:55:29,631 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,631 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:29,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:29,632 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,632 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:29,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:29,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:29,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:29,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:55:29,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79bc3d8e[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:29,633 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:55:29,634 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,635 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,636 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:55:29,636 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:55:29,636 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:29,636 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,636 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:29,637 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:55:29,637 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,637 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:29,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:29,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:29,637 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:55:29,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:29,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:29,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:55:29,640 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:55:29,640 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:29,640 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:29,640 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,640 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:29,642 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,642 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,642 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:29,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:29,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:29,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:29,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:29,643 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,644 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:29,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:29,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:29,648 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:29,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:29,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:29,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:29,648 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:29,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:29,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:29,648 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,649 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,649 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,649 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:29,649 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:29,649 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:29,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:29,650 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:55:29,749 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:29,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:29,754 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:29,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:29,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,758 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:29,758 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:29,758 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:55:29,758 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:55:29,759 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,759 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,760 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,760 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:29,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:29,760 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,760 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:29,761 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:29,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,762 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,762 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:55:29,762 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:29,762 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:55:29,762 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:29,762 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:55:29,762 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:55:29,762 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:29,762 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:29,762 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. started... 2023-03-24 21:55:29,763 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b21dad992cbb40d94473c32040a2d9a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:55:29,785 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103126bae91abde4660b86ee8e08e381c46 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103126bae91abde4660b86ee8e08e381c46 2023-03-24 21:55:30,124 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-24 21:55:30,190 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/5a023377a1ad45d7ac3ef60aa53fb581 2023-03-24 21:55:30,195 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/5a023377a1ad45d7ac3ef60aa53fb581 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/5a023377a1ad45d7ac3ef60aa53fb581 2023-03-24 21:55:30,200 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/5a023377a1ad45d7ac3ef60aa53fb581, entries=1, sequenceid=33, filesize=4.9 K 2023-03-24 21:55:30,201 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b21dad992cbb40d94473c32040a2d9a in 438ms, sequenceid=33, compaction requested=true 2023-03-24 21:55:30,201 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:30,201 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:30,201 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:55:30,201 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:55:30,201 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,201 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:55:30,201 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:55:30,204 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,204 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:30,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:30,205 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,205 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:30,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:30,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:30,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:30,207 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:55:30,207 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66b25587[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:30,207 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:55:30,207 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,208 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,208 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:55:30,208 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:55:30,208 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:30,208 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,209 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:30,211 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:55:30,211 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:30,211 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,211 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:55:30,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:30,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:30,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:30,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:30,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:30,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:30,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:55:30,214 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:55:30,214 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:30,214 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:30,214 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,214 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:30,215 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,215 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,215 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:30,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:30,216 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:30,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:30,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:30,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:30,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:30,217 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:30,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,223 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,223 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:30,223 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:30,223 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:30,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:30,223 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:30,223 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,223 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,223 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,223 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,223 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:30,223 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:30,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:30,223 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:30,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:55:30,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:30,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:30,324 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:30,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:30,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:30,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:30,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,331 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:30,331 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:30,331 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:55:30,331 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:55:30,332 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,332 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,334 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:30,334 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:30,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:30,335 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:30,335 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:30,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,335 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:30,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,336 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,336 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:55:30,336 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:30,336 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:55:30,336 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:30,336 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:55:30,336 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:55:30,336 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:30,336 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:30,336 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. started... 2023-03-24 21:55:30,337 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b21dad992cbb40d94473c32040a2d9a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:55:30,749 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907312edb8278694d2e9713f0c0c49d7f1e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907312edb8278694d2e9713f0c0c49d7f1e 2023-03-24 21:55:31,156 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/0f389d5a2d2d43fba2fd0f401ccb28df 2023-03-24 21:55:31,161 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/0f389d5a2d2d43fba2fd0f401ccb28df as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/0f389d5a2d2d43fba2fd0f401ccb28df 2023-03-24 21:55:31,165 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/0f389d5a2d2d43fba2fd0f401ccb28df, entries=1, sequenceid=37, filesize=4.9 K 2023-03-24 21:55:31,166 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b21dad992cbb40d94473c32040a2d9a in 829ms, sequenceid=37, compaction requested=true 2023-03-24 21:55:31,166 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:31,166 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:31,166 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:55:31,166 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:55:31,166 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,166 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:55:31,166 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:55:31,168 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,168 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:31,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:31,168 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,168 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:31,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:31,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:31,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:31,171 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:55:31,171 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@322cd602[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:31,171 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:55:31,171 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,173 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,173 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:55:31,173 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,173 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:31,173 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:55:31,173 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:31,175 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,175 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:55:31,175 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:31,175 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:55:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:31,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:31,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:55:31,178 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:55:31,178 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:31,178 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:31,178 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,178 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:31,180 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,180 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,180 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:31,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:31,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:31,180 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:31,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,181 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:31,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,183 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:31,183 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,183 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:31,183 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:55:31,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:31,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:31,183 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:31,183 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:31,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:31,184 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,184 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:31,184 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:31,184 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,184 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:31,184 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:31,184 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,284 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:31,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:31,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:31,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:31,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,291 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:31,291 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:31,291 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-24 21:55:31,291 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-24 21:55:31,291 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,291 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,293 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,293 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:31,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:31,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:31,293 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:31,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,293 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:31,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:31,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,294 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,294 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-24 21:55:31,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:31,294 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-24 21:55:31,294 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:31,295 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-24 21:55:31,295 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-24 21:55:31,295 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:31,295 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:31,295 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. started... 2023-03-24 21:55:31,295 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b21dad992cbb40d94473c32040a2d9a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-24 21:55:31,709 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120d5aab1ea44c64be4a02f59526fba9039 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120d5aab1ea44c64be4a02f59526fba9039 2023-03-24 21:55:32,114 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/5bdb416cd0db4a7cb868f5df047b1e56 2023-03-24 21:55:32,119 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/5bdb416cd0db4a7cb868f5df047b1e56 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/5bdb416cd0db4a7cb868f5df047b1e56 2023-03-24 21:55:32,123 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/5bdb416cd0db4a7cb868f5df047b1e56, entries=1, sequenceid=41, filesize=4.9 K 2023-03-24 21:55:32,124 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b21dad992cbb40d94473c32040a2d9a in 829ms, sequenceid=41, compaction requested=true 2023-03-24 21:55:32,125 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:32,125 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:32,125 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-24 21:55:32,125 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-24 21:55:32,125 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,125 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-24 21:55:32,125 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-24 21:55:32,126 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,127 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:32,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:32,127 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,127 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:32,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:32,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:32,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:32,129 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-24 21:55:32,129 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7fb82d43[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:32,129 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-24 21:55:32,129 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,130 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,130 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-24 21:55:32,130 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-24 21:55:32,130 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:32,131 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,131 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:32,133 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-24 21:55:32,133 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:32,133 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-24 21:55:32,133 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:32,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:32,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:32,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:32,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:32,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:32,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-24 21:55:32,136 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-24 21:55:32,137 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:32,137 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:32,137 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,137 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:32,138 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:32,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:32,139 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,139 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:32,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,139 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:32,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:32,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:32,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:32,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,140 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:32,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,149 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,149 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,149 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,149 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:32,149 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:32,149 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-24 21:55:32,149 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:32,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:32,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-24 21:55:32,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:32,149 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:32,149 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,149 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:32,149 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,149 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:32,150 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:32,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:32,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:32,250 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:32,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:32,252 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-03-24 21:55:32,253 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-24 21:55:32,254 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-24 21:55:32,255 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695532255 2023-03-24 21:55:32,255 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:55:32,256 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-24 21:55:32,257 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-24 21:55:32,257 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:55:32,257 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x64affd45 to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:55:32,264 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79722a70, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:55:32,264 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:55:33,105 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103aefe068f6bd341e2b59044ad34ea9776 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103aefe068f6bd341e2b59044ad34ea9776 2023-03-24 21:55:33,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:55:33,120 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:55:33,122 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:59002, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:55:33,136 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:55:33,138 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:41146, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:55:33,148 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/8bd2672d664f469f8d0ce0f23c6706b8 first=Optional[r5] last=Optional[r6] 2023-03-24 21:55:33,149 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/8bd2672d664f469f8d0ce0f23c6706b8}] 2023-03-24 21:55:33,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/8bd2672d664f469f8d0ce0f23c6706b8 for inclusion in 2b21dad992cbb40d94473c32040a2d9a/family1 2023-03-24 21:55:33,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-24 21:55:33,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:55:33,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:33,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/8bd2672d664f469f8d0ce0f23c6706b8 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hnlc5hcdc81pt7gg3s508p4t2lekk3fsla01ko1o11i7p5hs3ev44dh56nop80g/family1/8bd2672d664f469f8d0ce0f23c6706b8 2023-03-24 21:55:33,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hnlc5hcdc81pt7gg3s508p4t2lekk3fsla01ko1o11i7p5hs3ev44dh56nop80g/family1/8bd2672d664f469f8d0ce0f23c6706b8 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3152f77569d04d4f9d8018c128c44343_SeqId_44_ 2023-03-24 21:55:33,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hnlc5hcdc81pt7gg3s508p4t2lekk3fsla01ko1o11i7p5hs3ev44dh56nop80g/family1/8bd2672d664f469f8d0ce0f23c6706b8 into 2b21dad992cbb40d94473c32040a2d9a/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3152f77569d04d4f9d8018c128c44343_SeqId_44_ - updating store file list. 2023-03-24 21:55:33,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3152f77569d04d4f9d8018c128c44343_SeqId_44_ into 2b21dad992cbb40d94473c32040a2d9a/family1 2023-03-24 21:55:33,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hnlc5hcdc81pt7gg3s508p4t2lekk3fsla01ko1o11i7p5hs3ev44dh56nop80g/family1/8bd2672d664f469f8d0ce0f23c6706b8 into 2b21dad992cbb40d94473c32040a2d9a/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3152f77569d04d4f9d8018c128c44343_SeqId_44_) 2023-03-24 21:55:33,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hnlc5hcdc81pt7gg3s508p4t2lekk3fsla01ko1o11i7p5hs3ev44dh56nop80g/family1/8bd2672d664f469f8d0ce0f23c6706b8 2023-03-24 21:55:33,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-24 21:55:33,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-24 21:55:33,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(1912): 2b21dad992cbb40d94473c32040a2d9a/family1 is initiating minor compaction 2023-03-24 21:55:33,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@17e13a7d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:55:33,167 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2b21dad992cbb40d94473c32040a2d9a/family1 in testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:33,167 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/646897bb59ec4b7f8712ede9d306ca12, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/143547e18f1f44e78635d377894b5f46, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/0c2f43e5d2e846cf9bd6e3ea21bc730b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/c9f13fa77349465faea350d294067b64, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/a4409e238bbb4bb686ff1e14eefb0e56, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3f36365f683d43d0a7c74bffdb79aeda, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/5a023377a1ad45d7ac3ef60aa53fb581, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/0f389d5a2d2d43fba2fd0f401ccb28df, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/5bdb416cd0db4a7cb868f5df047b1e56, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3152f77569d04d4f9d8018c128c44343_SeqId_44_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp, totalSize=49.7 K 2023-03-24 21:55:33,167 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 646897bb59ec4b7f8712ede9d306ca12, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-24 21:55:33,167 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 143547e18f1f44e78635d377894b5f46, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-24 21:55:33,167 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 0c2f43e5d2e846cf9bd6e3ea21bc730b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-24 21:55:33,168 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting c9f13fa77349465faea350d294067b64, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-24 21:55:33,168 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting a4409e238bbb4bb686ff1e14eefb0e56, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-24 21:55:33,168 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 3f36365f683d43d0a7c74bffdb79aeda, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-24 21:55:33,168 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 5a023377a1ad45d7ac3ef60aa53fb581, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-24 21:55:33,168 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 0f389d5a2d2d43fba2fd0f401ccb28df, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-24 21:55:33,168 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 5bdb416cd0db4a7cb868f5df047b1e56, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-24 21:55:33,168 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 3152f77569d04d4f9d8018c128c44343_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-24 21:55:33,169 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:55:33,170 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:33,171 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010179c6b23c36364179bc619bb0056b1d6f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010179c6b23c36364179bc619bb0056b1d6f 2023-03-24 21:55:33,174 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103126bae91abde4660b86ee8e08e381c46 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103126bae91abde4660b86ee8e08e381c46 2023-03-24 21:55:33,174 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-24 21:55:33,175 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-24 21:55:33,175 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:55:33,175 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:55:33,223 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2b21dad992cbb40d94473c32040a2d9a#family1#compaction#165 average throughput is 0.11 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-03-24 21:55:33,261 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/20cbb626ea8d43ea830a0b21e7baff26 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/20cbb626ea8d43ea830a0b21e7baff26 2023-03-24 21:55:33,266 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 2b21dad992cbb40d94473c32040a2d9a/family1 of 2b21dad992cbb40d94473c32040a2d9a into 20cbb626ea8d43ea830a0b21e7baff26(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:33,266 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:33,266 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1679694933167; duration=0sec 2023-03-24 21:55:33,267 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:55:33,297 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/646897bb59ec4b7f8712ede9d306ca12, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/143547e18f1f44e78635d377894b5f46, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/0c2f43e5d2e846cf9bd6e3ea21bc730b, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/c9f13fa77349465faea350d294067b64, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/a4409e238bbb4bb686ff1e14eefb0e56, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3f36365f683d43d0a7c74bffdb79aeda, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/5a023377a1ad45d7ac3ef60aa53fb581, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/0f389d5a2d2d43fba2fd0f401ccb28df, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/5bdb416cd0db4a7cb868f5df047b1e56, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3152f77569d04d4f9d8018c128c44343_SeqId_44_] to archive 2023-03-24 21:55:33,298 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:33,301 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/646897bb59ec4b7f8712ede9d306ca12 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/646897bb59ec4b7f8712ede9d306ca12 2023-03-24 21:55:33,303 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/143547e18f1f44e78635d377894b5f46 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/143547e18f1f44e78635d377894b5f46 2023-03-24 21:55:33,305 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/0c2f43e5d2e846cf9bd6e3ea21bc730b to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/0c2f43e5d2e846cf9bd6e3ea21bc730b 2023-03-24 21:55:33,307 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/c9f13fa77349465faea350d294067b64 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/c9f13fa77349465faea350d294067b64 2023-03-24 21:55:33,309 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/a4409e238bbb4bb686ff1e14eefb0e56 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/a4409e238bbb4bb686ff1e14eefb0e56 2023-03-24 21:55:33,310 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3f36365f683d43d0a7c74bffdb79aeda to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3f36365f683d43d0a7c74bffdb79aeda 2023-03-24 21:55:33,312 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/5a023377a1ad45d7ac3ef60aa53fb581 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/5a023377a1ad45d7ac3ef60aa53fb581 2023-03-24 21:55:33,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/0f389d5a2d2d43fba2fd0f401ccb28df to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/0f389d5a2d2d43fba2fd0f401ccb28df 2023-03-24 21:55:33,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/5bdb416cd0db4a7cb868f5df047b1e56 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/5bdb416cd0db4a7cb868f5df047b1e56 2023-03-24 21:55:33,318 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3152f77569d04d4f9d8018c128c44343_SeqId_44_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/3152f77569d04d4f9d8018c128c44343_SeqId_44_ 2023-03-24 21:55:33,664 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512312d76787dc6c54677bc88b406d03d84b3 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512312d76787dc6c54677bc88b406d03d84b3 2023-03-24 21:55:33,665 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:55:33,691 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/4d07862f46b04bfba47de2e196d03cfb first=Optional[r1] last=Optional[r4] 2023-03-24 21:55:33,692 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/4d07862f46b04bfba47de2e196d03cfb}] 2023-03-24 21:55:33,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/4d07862f46b04bfba47de2e196d03cfb for inclusion in 2b21dad992cbb40d94473c32040a2d9a/family1 2023-03-24 21:55:33,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-24 21:55:33,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:55:33,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:33,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/4d07862f46b04bfba47de2e196d03cfb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__sjs1ssifgcivtnpt806d694hp84u2shvid7nj1kt6jco98j6avld21putvb7r64o/family1/4d07862f46b04bfba47de2e196d03cfb 2023-03-24 21:55:33,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__sjs1ssifgcivtnpt806d694hp84u2shvid7nj1kt6jco98j6avld21putvb7r64o/family1/4d07862f46b04bfba47de2e196d03cfb as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/b02940ab8c734fd7a5bc7ddd6d6c3c29_SeqId_47_ 2023-03-24 21:55:33,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__sjs1ssifgcivtnpt806d694hp84u2shvid7nj1kt6jco98j6avld21putvb7r64o/family1/4d07862f46b04bfba47de2e196d03cfb into 2b21dad992cbb40d94473c32040a2d9a/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/b02940ab8c734fd7a5bc7ddd6d6c3c29_SeqId_47_ - updating store file list. 2023-03-24 21:55:33,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/b02940ab8c734fd7a5bc7ddd6d6c3c29_SeqId_47_ into 2b21dad992cbb40d94473c32040a2d9a/family1 2023-03-24 21:55:33,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__sjs1ssifgcivtnpt806d694hp84u2shvid7nj1kt6jco98j6avld21putvb7r64o/family1/4d07862f46b04bfba47de2e196d03cfb into 2b21dad992cbb40d94473c32040a2d9a/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/b02940ab8c734fd7a5bc7ddd6d6c3c29_SeqId_47_) 2023-03-24 21:55:33,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__sjs1ssifgcivtnpt806d694hp84u2shvid7nj1kt6jco98j6avld21putvb7r64o/family1/4d07862f46b04bfba47de2e196d03cfb 2023-03-24 21:55:33,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-24 21:55:33,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-24 21:55:33,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(1912): 2b21dad992cbb40d94473c32040a2d9a/family1 is initiating minor compaction (all files) 2023-03-24 21:55:33,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@790cee9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:55:33,710 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2b21dad992cbb40d94473c32040a2d9a/family1 in testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:33,710 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/79c64a3156324c919a4da2158e4a5156, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/20cbb626ea8d43ea830a0b21e7baff26, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/b02940ab8c734fd7a5bc7ddd6d6c3c29_SeqId_47_] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp, totalSize=178.1 K 2023-03-24 21:55:33,711 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 79c64a3156324c919a4da2158e4a5156, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841999 2023-03-24 21:55:33,711 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting 20cbb626ea8d43ea830a0b21e7baff26, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241999 2023-03-24 21:55:33,711 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] compactions.Compactor(207): Compacting b02940ab8c734fd7a5bc7ddd6d6c3c29_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965241999 2023-03-24 21:55:33,712 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:55:33,713 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:33,715 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201071a0628cce74485aa872039228cd891 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201071a0628cce74485aa872039228cd891 2023-03-24 21:55:33,722 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205f3475b2ed08f49f78ba6e77e0305451f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205f3475b2ed08f49f78ba6e77e0305451f 2023-03-24 21:55:33,724 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228fa7748b7992447838107aab21a9dc051 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228fa7748b7992447838107aab21a9dc051 2023-03-24 21:55:33,733 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123124bdb7cba8964770af77d2fc217cb1f4 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123124bdb7cba8964770af77d2fc217cb1f4 2023-03-24 21:55:33,733 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-24 21:55:33,733 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-24 21:55:33,738 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:55:33,738 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:55:33,753 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2b21dad992cbb40d94473c32040a2d9a#family1#compaction#166 average throughput is 5.56 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-03-24 21:55:33,858 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130b5791abcfdb5439694821c3f0d49b0bd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130b5791abcfdb5439694821c3f0d49b0bd 2023-03-24 21:55:33,859 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:55:33,923 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b26813a66f834b9ca6a9057a63d14882 first=Optional[r00] last=Optional[r8] 2023-03-24 21:55:33,925 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b26813a66f834b9ca6a9057a63d14882}] 2023-03-24 21:55:33,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b26813a66f834b9ca6a9057a63d14882 for inclusion in 2b21dad992cbb40d94473c32040a2d9a/family1 2023-03-24 21:55:33,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-24 21:55:33,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last= 2023-03-24 21:55:33,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:33,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b26813a66f834b9ca6a9057a63d14882 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hn14snbbdkbbav3r0u4vlknechbd2ht50rte5ao0mm9ch3mc6ls6rdrhbcj69a86/family1/b26813a66f834b9ca6a9057a63d14882 2023-03-24 21:55:33,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hn14snbbdkbbav3r0u4vlknechbd2ht50rte5ao0mm9ch3mc6ls6rdrhbcj69a86/family1/b26813a66f834b9ca6a9057a63d14882 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/79b7eea1aec74de8b2f9250bd8636523_SeqId_49_ 2023-03-24 21:55:33,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hn14snbbdkbbav3r0u4vlknechbd2ht50rte5ao0mm9ch3mc6ls6rdrhbcj69a86/family1/b26813a66f834b9ca6a9057a63d14882 into 2b21dad992cbb40d94473c32040a2d9a/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/79b7eea1aec74de8b2f9250bd8636523_SeqId_49_ - updating store file list. 2023-03-24 21:55:33,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/79b7eea1aec74de8b2f9250bd8636523_SeqId_49_ into 2b21dad992cbb40d94473c32040a2d9a/family1 2023-03-24 21:55:33,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hn14snbbdkbbav3r0u4vlknechbd2ht50rte5ao0mm9ch3mc6ls6rdrhbcj69a86/family1/b26813a66f834b9ca6a9057a63d14882 into 2b21dad992cbb40d94473c32040a2d9a/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/79b7eea1aec74de8b2f9250bd8636523_SeqId_49_) 2023-03-24 21:55:33,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hn14snbbdkbbav3r0u4vlknechbd2ht50rte5ao0mm9ch3mc6ls6rdrhbcj69a86/family1/b26813a66f834b9ca6a9057a63d14882 2023-03-24 21:55:33,948 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:55:33,948 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:33,950 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120d5aab1ea44c64be4a02f59526fba9039 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120d5aab1ea44c64be4a02f59526fba9039 2023-03-24 21:55:33,951 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128fe687c9daba24876b43ef9b41dc4f060 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128fe687c9daba24876b43ef9b41dc4f060 2023-03-24 21:55:33,952 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113006edd780d1494652b78259edd9897a3c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113006edd780d1494652b78259edd9897a3c 2023-03-24 21:55:33,952 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-24 21:55:33,952 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-24 21:55:33,953 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:55:33,953 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:55:33,953 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-24 21:55:33,953 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-24 21:55:33,953 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:55:33,956 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-24 21:55:33,957 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.7040 sec 2023-03-24 21:55:34,018 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:55:34,023 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64affd45 to 127.0.0.1:54680 2023-03-24 21:55:34,023 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:34,023 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:55:34,023 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x62df6183 to 127.0.0.1:54680 2023-03-24 21:55:34,023 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:34,024 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:55:34,024 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x27ca7234 to 127.0.0.1:54680 2023-03-24 21:55:34,024 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:34,240 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/.tmp/family1/a7aad531deaf4578875a446f84551e7e as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/a7aad531deaf4578875a446f84551e7e 2023-03-24 21:55:34,245 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 2b21dad992cbb40d94473c32040a2d9a/family1 of 2b21dad992cbb40d94473c32040a2d9a into a7aad531deaf4578875a446f84551e7e(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:34,245 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:34,245 INFO [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1679694933710; duration=0sec 2023-03-24 21:55:34,245 DEBUG [RS:0;jenkins-hbase9:41807-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-24 21:55:34,275 INFO [Listener at localhost/39907] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=248 (was 253), OpenFileDescriptor=633 (was 657), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=395 (was 338) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=7851 (was 7910) 2023-03-24 21:55:34,295 INFO [Listener at localhost/39907] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=248, OpenFileDescriptor=633, MaxFileDescriptor=60000, SystemLoadAverage=395, ProcessCount=169, AvailableMemoryMB=7849 2023-03-24 21:55:34,295 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/79c64a3156324c919a4da2158e4a5156, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/20cbb626ea8d43ea830a0b21e7baff26, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/b02940ab8c734fd7a5bc7ddd6d6c3c29_SeqId_47_] to archive 2023-03-24 21:55:34,296 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:34,298 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/79c64a3156324c919a4da2158e4a5156 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/79c64a3156324c919a4da2158e4a5156 2023-03-24 21:55:34,300 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/20cbb626ea8d43ea830a0b21e7baff26 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/20cbb626ea8d43ea830a0b21e7baff26 2023-03-24 21:55:34,301 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/b02940ab8c734fd7a5bc7ddd6d6c3c29_SeqId_47_ to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/family1/b02940ab8c734fd7a5bc7ddd6d6c3c29_SeqId_47_ 2023-03-24 21:55:34,320 DEBUG [Listener at localhost/39907] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-03-24 21:55:34,394 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.HMaster$4(2112): Client=jenkins//172.31.2.10 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCFV\x05\xAC6\xDB:\xEB\xDA\xBE\xF0P\xA8\xF6p\xB4"\x10\x86\xB8S\xB9kg)\x11\xEE>\x01T\xB7Ns\x80*\x10\x13\xC8\xD3\x04\xBD$X\xB6\xC2V\x1EN\xC9|2\x132\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-24 21:55:34,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-03-24 21:55:34,396 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-03-24 21:55:34,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-03-24 21:55:34,397 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-24 21:55:34,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-24 21:55:34,400 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a 2023-03-24 21:55:34,400 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48 2023-03-24 21:55:34,400 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24 2023-03-24 21:55:34,401 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24 empty. 2023-03-24 21:55:34,401 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a empty. 2023-03-24 21:55:34,401 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48 empty. 2023-03-24 21:55:34,401 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24 2023-03-24 21:55:34,401 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a 2023-03-24 21:55:34,401 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48 2023-03-24 21:55:34,401 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-03-24 21:55:34,414 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-03-24 21:55:34,415 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 845f335cfe6009956f8aee7e54864e24, NAME => 'testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCFV\x05\xAC6\xDB:\xEB\xDA\xBE\xF0P\xA8\xF6p\xB4"\x10\x86\xB8S\xB9kg)\x11\xEE>\x01T\xB7Ns\x80*\x10\x13\xC8\xD3\x04\xBD$X\xB6\xC2V\x1EN\xC9|2\x132\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:55:34,416 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => cc01e67bbfbc97fe3b57bdba3619e55a, NAME => 'testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCFV\x05\xAC6\xDB:\xEB\xDA\xBE\xF0P\xA8\xF6p\xB4"\x10\x86\xB8S\xB9kg)\x11\xEE>\x01T\xB7Ns\x80*\x10\x13\xC8\xD3\x04\xBD$X\xB6\xC2V\x1EN\xC9|2\x132\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:55:34,416 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => d12810111afaf6de0915ac2e031e6b48, NAME => 'testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCFV\x05\xAC6\xDB:\xEB\xDA\xBE\xF0P\xA8\xF6p\xB4"\x10\x86\xB8S\xB9kg)\x11\xEE>\x01T\xB7Ns\x80*\x10\x13\xC8\xD3\x04\xBD$X\xB6\xC2V\x1EN\xC9|2\x132\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/.tmp 2023-03-24 21:55:34,430 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:34,430 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 845f335cfe6009956f8aee7e54864e24, disabling compactions & flushes 2023-03-24 21:55:34,430 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:34,430 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:34,430 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. after waiting 0 ms 2023-03-24 21:55:34,430 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:34,430 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:34,430 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 845f335cfe6009956f8aee7e54864e24: 2023-03-24 21:55:34,431 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:34,431 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing cc01e67bbfbc97fe3b57bdba3619e55a, disabling compactions & flushes 2023-03-24 21:55:34,431 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:34,431 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:34,431 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. after waiting 0 ms 2023-03-24 21:55:34,431 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:34,431 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:34,431 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for cc01e67bbfbc97fe3b57bdba3619e55a: 2023-03-24 21:55:34,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-24 21:55:34,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-24 21:55:34,832 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:34,832 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing d12810111afaf6de0915ac2e031e6b48, disabling compactions & flushes 2023-03-24 21:55:34,832 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:34,832 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:34,832 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. after waiting 0 ms 2023-03-24 21:55:34,832 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:34,832 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:34,832 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for d12810111afaf6de0915ac2e031e6b48: 2023-03-24 21:55:34,834 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-03-24 21:55:34,835 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694934835"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694934835"}]},"ts":"1679694934835"} 2023-03-24 21:55:34,835 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694934835"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694934835"}]},"ts":"1679694934835"} 2023-03-24 21:55:34,835 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1679694934835"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679694934835"}]},"ts":"1679694934835"} 2023-03-24 21:55:34,837 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-24 21:55:34,837 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-24 21:55:34,838 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694934838"}]},"ts":"1679694934838"} 2023-03-24 21:55:34,838 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-03-24 21:55:34,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cc01e67bbfbc97fe3b57bdba3619e55a, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d12810111afaf6de0915ac2e031e6b48, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=845f335cfe6009956f8aee7e54864e24, ASSIGN}] 2023-03-24 21:55:34,843 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cc01e67bbfbc97fe3b57bdba3619e55a, ASSIGN 2023-03-24 21:55:34,843 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=845f335cfe6009956f8aee7e54864e24, ASSIGN 2023-03-24 21:55:34,843 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d12810111afaf6de0915ac2e031e6b48, ASSIGN 2023-03-24 21:55:34,844 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cc01e67bbfbc97fe3b57bdba3619e55a, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:55:34,844 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d12810111afaf6de0915ac2e031e6b48, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:55:34,844 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=845f335cfe6009956f8aee7e54864e24, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41807,1679694819103; forceNewPlan=false, retain=false 2023-03-24 21:55:34,996 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=845f335cfe6009956f8aee7e54864e24, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:34,996 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=cc01e67bbfbc97fe3b57bdba3619e55a, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:34,996 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=d12810111afaf6de0915ac2e031e6b48, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:34,996 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694934996"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694934996"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694934996"}]},"ts":"1679694934996"} 2023-03-24 21:55:34,996 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694934996"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694934996"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694934996"}]},"ts":"1679694934996"} 2023-03-24 21:55:34,996 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1679694934996"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679694934996"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679694934996"}]},"ts":"1679694934996"} 2023-03-24 21:55:34,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure cc01e67bbfbc97fe3b57bdba3619e55a, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:34,998 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=119, state=RUNNABLE; OpenRegionProcedure 845f335cfe6009956f8aee7e54864e24, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:34,999 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=118, state=RUNNABLE; OpenRegionProcedure d12810111afaf6de0915ac2e031e6b48, server=jenkins-hbase9.apache.org,41807,1679694819103}] 2023-03-24 21:55:35,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-24 21:55:35,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:35,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 845f335cfe6009956f8aee7e54864e24, NAME => 'testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24.', STARTKEY => '2', ENDKEY => ''} 2023-03-24 21:55:35,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 845f335cfe6009956f8aee7e54864e24 2023-03-24 21:55:35,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:35,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for 845f335cfe6009956f8aee7e54864e24 2023-03-24 21:55:35,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for 845f335cfe6009956f8aee7e54864e24 2023-03-24 21:55:35,174 INFO [StoreOpener-845f335cfe6009956f8aee7e54864e24-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 845f335cfe6009956f8aee7e54864e24 2023-03-24 21:55:35,175 DEBUG [StoreOpener-845f335cfe6009956f8aee7e54864e24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1 2023-03-24 21:55:35,175 DEBUG [StoreOpener-845f335cfe6009956f8aee7e54864e24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1 2023-03-24 21:55:35,176 INFO [StoreOpener-845f335cfe6009956f8aee7e54864e24-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 845f335cfe6009956f8aee7e54864e24 columnFamilyName family1 2023-03-24 21:55:35,177 INFO [StoreOpener-845f335cfe6009956f8aee7e54864e24-1] regionserver.HStore(310): Store=845f335cfe6009956f8aee7e54864e24/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:35,177 INFO [StoreOpener-845f335cfe6009956f8aee7e54864e24-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 845f335cfe6009956f8aee7e54864e24 2023-03-24 21:55:35,178 DEBUG [StoreOpener-845f335cfe6009956f8aee7e54864e24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2 2023-03-24 21:55:35,178 DEBUG [StoreOpener-845f335cfe6009956f8aee7e54864e24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2 2023-03-24 21:55:35,179 INFO [StoreOpener-845f335cfe6009956f8aee7e54864e24-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 845f335cfe6009956f8aee7e54864e24 columnFamilyName family2 2023-03-24 21:55:35,180 INFO [StoreOpener-845f335cfe6009956f8aee7e54864e24-1] regionserver.HStore(310): Store=845f335cfe6009956f8aee7e54864e24/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:35,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24 2023-03-24 21:55:35,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24 2023-03-24 21:55:35,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:55:35,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for 845f335cfe6009956f8aee7e54864e24 2023-03-24 21:55:35,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:55:35,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened 845f335cfe6009956f8aee7e54864e24; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70178888, jitterRate=0.04574692249298096}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:55:35,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for 845f335cfe6009956f8aee7e54864e24: 2023-03-24 21:55:35,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24., pid=121, masterSystemTime=1679694935149 2023-03-24 21:55:35,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:35,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:35,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:35,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d12810111afaf6de0915ac2e031e6b48, NAME => 'testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48.', STARTKEY => '1', ENDKEY => '2'} 2023-03-24 21:55:35,186 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=845f335cfe6009956f8aee7e54864e24, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:35,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin d12810111afaf6de0915ac2e031e6b48 2023-03-24 21:55:35,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:35,187 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694935186"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694935186"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694935186"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694935186"}]},"ts":"1679694935186"} 2023-03-24 21:55:35,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for d12810111afaf6de0915ac2e031e6b48 2023-03-24 21:55:35,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for d12810111afaf6de0915ac2e031e6b48 2023-03-24 21:55:35,189 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-03-24 21:55:35,189 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; OpenRegionProcedure 845f335cfe6009956f8aee7e54864e24, server=jenkins-hbase9.apache.org,41807,1679694819103 in 190 msec 2023-03-24 21:55:35,190 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=845f335cfe6009956f8aee7e54864e24, ASSIGN in 348 msec 2023-03-24 21:55:35,207 INFO [StoreOpener-d12810111afaf6de0915ac2e031e6b48-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d12810111afaf6de0915ac2e031e6b48 2023-03-24 21:55:35,208 DEBUG [StoreOpener-d12810111afaf6de0915ac2e031e6b48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1 2023-03-24 21:55:35,209 DEBUG [StoreOpener-d12810111afaf6de0915ac2e031e6b48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1 2023-03-24 21:55:35,209 INFO [StoreOpener-d12810111afaf6de0915ac2e031e6b48-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d12810111afaf6de0915ac2e031e6b48 columnFamilyName family1 2023-03-24 21:55:35,210 INFO [StoreOpener-d12810111afaf6de0915ac2e031e6b48-1] regionserver.HStore(310): Store=d12810111afaf6de0915ac2e031e6b48/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:35,210 INFO [StoreOpener-d12810111afaf6de0915ac2e031e6b48-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d12810111afaf6de0915ac2e031e6b48 2023-03-24 21:55:35,211 DEBUG [StoreOpener-d12810111afaf6de0915ac2e031e6b48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2 2023-03-24 21:55:35,211 DEBUG [StoreOpener-d12810111afaf6de0915ac2e031e6b48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2 2023-03-24 21:55:35,211 INFO [StoreOpener-d12810111afaf6de0915ac2e031e6b48-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d12810111afaf6de0915ac2e031e6b48 columnFamilyName family2 2023-03-24 21:55:35,212 INFO [StoreOpener-d12810111afaf6de0915ac2e031e6b48-1] regionserver.HStore(310): Store=d12810111afaf6de0915ac2e031e6b48/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:35,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48 2023-03-24 21:55:35,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48 2023-03-24 21:55:35,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:55:35,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for d12810111afaf6de0915ac2e031e6b48 2023-03-24 21:55:35,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:55:35,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened d12810111afaf6de0915ac2e031e6b48; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71452017, jitterRate=0.06471802294254303}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:55:35,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for d12810111afaf6de0915ac2e031e6b48: 2023-03-24 21:55:35,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48., pid=122, masterSystemTime=1679694935149 2023-03-24 21:55:35,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:35,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:35,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:35,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cc01e67bbfbc97fe3b57bdba3619e55a, NAME => 'testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a.', STARTKEY => '', ENDKEY => '1'} 2023-03-24 21:55:35,218 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=d12810111afaf6de0915ac2e031e6b48, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:35,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin cc01e67bbfbc97fe3b57bdba3619e55a 2023-03-24 21:55:35,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-24 21:55:35,218 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1679694935218"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694935218"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694935218"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694935218"}]},"ts":"1679694935218"} 2023-03-24 21:55:35,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7894): checking encryption for cc01e67bbfbc97fe3b57bdba3619e55a 2023-03-24 21:55:35,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7897): checking classloading for cc01e67bbfbc97fe3b57bdba3619e55a 2023-03-24 21:55:35,220 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=118 2023-03-24 21:55:35,220 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=118, state=SUCCESS; OpenRegionProcedure d12810111afaf6de0915ac2e031e6b48, server=jenkins-hbase9.apache.org,41807,1679694819103 in 220 msec 2023-03-24 21:55:35,221 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d12810111afaf6de0915ac2e031e6b48, ASSIGN in 379 msec 2023-03-24 21:55:35,240 INFO [StoreOpener-cc01e67bbfbc97fe3b57bdba3619e55a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cc01e67bbfbc97fe3b57bdba3619e55a 2023-03-24 21:55:35,242 DEBUG [StoreOpener-cc01e67bbfbc97fe3b57bdba3619e55a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1 2023-03-24 21:55:35,242 DEBUG [StoreOpener-cc01e67bbfbc97fe3b57bdba3619e55a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1 2023-03-24 21:55:35,242 INFO [StoreOpener-cc01e67bbfbc97fe3b57bdba3619e55a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cc01e67bbfbc97fe3b57bdba3619e55a columnFamilyName family1 2023-03-24 21:55:35,243 INFO [StoreOpener-cc01e67bbfbc97fe3b57bdba3619e55a-1] regionserver.HStore(310): Store=cc01e67bbfbc97fe3b57bdba3619e55a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:35,243 INFO [StoreOpener-cc01e67bbfbc97fe3b57bdba3619e55a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region cc01e67bbfbc97fe3b57bdba3619e55a 2023-03-24 21:55:35,244 DEBUG [StoreOpener-cc01e67bbfbc97fe3b57bdba3619e55a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2 2023-03-24 21:55:35,244 DEBUG [StoreOpener-cc01e67bbfbc97fe3b57bdba3619e55a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2 2023-03-24 21:55:35,244 INFO [StoreOpener-cc01e67bbfbc97fe3b57bdba3619e55a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cc01e67bbfbc97fe3b57bdba3619e55a columnFamilyName family2 2023-03-24 21:55:35,245 INFO [StoreOpener-cc01e67bbfbc97fe3b57bdba3619e55a-1] regionserver.HStore(310): Store=cc01e67bbfbc97fe3b57bdba3619e55a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-24 21:55:35,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a 2023-03-24 21:55:35,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a 2023-03-24 21:55:35,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-24 21:55:35,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1055): writing seq id for cc01e67bbfbc97fe3b57bdba3619e55a 2023-03-24 21:55:35,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-24 21:55:35,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1072): Opened cc01e67bbfbc97fe3b57bdba3619e55a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64763471, jitterRate=-0.034949079155921936}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-24 21:55:35,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(965): Region open journal for cc01e67bbfbc97fe3b57bdba3619e55a: 2023-03-24 21:55:35,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a., pid=120, masterSystemTime=1679694935149 2023-03-24 21:55:35,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:35,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:35,251 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=cc01e67bbfbc97fe3b57bdba3619e55a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:35,251 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679694935251"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679694935251"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679694935251"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679694935251"}]},"ts":"1679694935251"} 2023-03-24 21:55:35,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-03-24 21:55:35,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure cc01e67bbfbc97fe3b57bdba3619e55a, server=jenkins-hbase9.apache.org,41807,1679694819103 in 254 msec 2023-03-24 21:55:35,254 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-03-24 21:55:35,254 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cc01e67bbfbc97fe3b57bdba3619e55a, ASSIGN in 412 msec 2023-03-24 21:55:35,255 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-24 21:55:35,255 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679694935255"}]},"ts":"1679694935255"} 2023-03-24 21:55:35,256 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-03-24 21:55:35,258 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-03-24 21:55:35,259 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 864 msec 2023-03-24 21:55:35,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-24 21:55:35,502 INFO [Listener at localhost/39907] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-03-24 21:55:35,507 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:35,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:35,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:35,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:35,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-24 21:55:35,511 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-24 21:55:35,511 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:35,511 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-24 21:55:35,511 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-24 21:55:35,512 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:35,512 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:35,515 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:35,515 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:35,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:35,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:35,515 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:35,515 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:35,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:35,516 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:35,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:35,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:35,516 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-24 21:55:35,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-24 21:55:35,516 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-24 21:55:35,516 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:35,516 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-24 21:55:35,517 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:35,517 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. started... 2023-03-24 21:55:35,517 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:55:35,517 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:35,517 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:35,517 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 845f335cfe6009956f8aee7e54864e24 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-24 21:55:35,517 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:35,517 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. started... 2023-03-24 21:55:35,517 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. started... 2023-03-24 21:55:35,518 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d12810111afaf6de0915ac2e031e6b48 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-24 21:55:35,518 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing cc01e67bbfbc97fe3b57bdba3619e55a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-24 21:55:35,567 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032483bdba911e76466d8dfca85a677433cc to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023032483bdba911e76466d8dfca85a677433cc 2023-03-24 21:55:35,567 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303246d8dcbdeae9b498c890bb6632dc53936 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303246d8dcbdeae9b498c890bb6632dc53936 2023-03-24 21:55:35,961 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032456e68d201c6e4c2da66b79e1bdd19dc1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032456e68d201c6e4c2da66b79e1bdd19dc1 2023-03-24 21:55:35,983 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family1/e4ea0ea6d14246b0998af1050b4977a6 2023-03-24 21:55:35,984 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family1/affc9d0aad9242beb3c6d2070b928b7d 2023-03-24 21:55:36,376 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family1/b4632e0ea5594ea0ae55575cf604ed56 2023-03-24 21:55:36,422 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324728909e39ef34a48b88f11aa9799b058 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230324728909e39ef34a48b88f11aa9799b058 2023-03-24 21:55:36,427 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324fbd2943e01284924b129567f0347ec1e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230324fbd2943e01284924b129567f0347ec1e 2023-03-24 21:55:36,812 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303245fc743b6dac449d2b356a5917c791358 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303245fc743b6dac449d2b356a5917c791358 2023-03-24 21:55:36,827 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family2/bc4bb3977d06473cbf5e84605b27363f 2023-03-24 21:55:36,830 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family1/affc9d0aad9242beb3c6d2070b928b7d as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/affc9d0aad9242beb3c6d2070b928b7d 2023-03-24 21:55:36,831 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family2/075c172d14b94eb389a89945e38ec7cd 2023-03-24 21:55:36,834 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family1/e4ea0ea6d14246b0998af1050b4977a6 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/e4ea0ea6d14246b0998af1050b4977a6 2023-03-24 21:55:36,854 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/affc9d0aad9242beb3c6d2070b928b7d, entries=4, sequenceid=5, filesize=5.5 K 2023-03-24 21:55:36,854 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family2/bc4bb3977d06473cbf5e84605b27363f as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/bc4bb3977d06473cbf5e84605b27363f 2023-03-24 21:55:36,856 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/e4ea0ea6d14246b0998af1050b4977a6, entries=4, sequenceid=5, filesize=5.5 K 2023-03-24 21:55:36,857 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family2/075c172d14b94eb389a89945e38ec7cd as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/075c172d14b94eb389a89945e38ec7cd 2023-03-24 21:55:36,858 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/bc4bb3977d06473cbf5e84605b27363f, entries=2, sequenceid=5, filesize=5.1 K 2023-03-24 21:55:36,858 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 845f335cfe6009956f8aee7e54864e24 in 1341ms, sequenceid=5, compaction requested=false 2023-03-24 21:55:36,858 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-24 21:55:36,859 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 845f335cfe6009956f8aee7e54864e24: 2023-03-24 21:55:36,859 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:36,859 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:55:36,860 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/075c172d14b94eb389a89945e38ec7cd, entries=2, sequenceid=5, filesize=5.1 K 2023-03-24 21:55:36,860 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for cc01e67bbfbc97fe3b57bdba3619e55a in 1342ms, sequenceid=5, compaction requested=false 2023-03-24 21:55:36,861 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for cc01e67bbfbc97fe3b57bdba3619e55a: 2023-03-24 21:55:36,861 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:36,861 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:55:37,217 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family2/38311ce68cb4477eb01f6a2d78fdaf12 2023-03-24 21:55:37,221 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family1/b4632e0ea5594ea0ae55575cf604ed56 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/b4632e0ea5594ea0ae55575cf604ed56 2023-03-24 21:55:37,245 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/b4632e0ea5594ea0ae55575cf604ed56, entries=4, sequenceid=5, filesize=5.5 K 2023-03-24 21:55:37,246 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family2/38311ce68cb4477eb01f6a2d78fdaf12 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/38311ce68cb4477eb01f6a2d78fdaf12 2023-03-24 21:55:37,249 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/38311ce68cb4477eb01f6a2d78fdaf12, entries=2, sequenceid=5, filesize=5.1 K 2023-03-24 21:55:37,250 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d12810111afaf6de0915ac2e031e6b48 in 1731ms, sequenceid=5, compaction requested=false 2023-03-24 21:55:37,250 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d12810111afaf6de0915ac2e031e6b48: 2023-03-24 21:55:37,250 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:37,250 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:55:37,250 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:55:37,250 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,250 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-24 21:55:37,250 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-24 21:55:37,251 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:37,251 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:37,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:37,252 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:37,252 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:37,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:37,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:37,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-24 21:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67f03f77[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:37,253 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-24 21:55:37,253 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:37,255 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:37,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:37,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:37,255 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-24 21:55:37,256 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-24 21:55:37,256 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:37,256 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,256 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:37,257 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-24 21:55:37,257 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:37,257 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,257 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-24 21:55:37,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:37,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:37,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:37,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:37,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:37,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:37,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:37,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-24 21:55:37,259 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-24 21:55:37,259 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:37,259 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:37,259 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-24 21:55:37,259 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:37,260 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:37,260 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:37,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:37,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:37,260 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:37,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:37,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:37,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:37,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:37,260 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:37,261 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:37,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:37,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,263 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,264 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:37,264 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:37,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-24 21:55:37,264 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:37,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:37,264 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:37,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:37,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:37,264 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:37,264 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:37,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:37,264 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-24 21:55:37,264 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:37,264 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:37,264 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:37,264 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:37,264 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:37,364 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:37,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:37,366 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:37,366 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:37,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:37,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:37,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:37,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-24 21:55:37,371 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-24 21:55:37,371 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:37,371 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-24 21:55:37,371 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-24 21:55:37,371 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:37,371 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:37,373 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,373 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:37,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:37,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:37,373 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:37,374 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:37,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:37,374 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-24 21:55:37,374 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-24 21:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-24 21:55:37,374 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-24 21:55:37,374 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:37,375 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-24 21:55:37,375 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:37,375 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. started... 2023-03-24 21:55:37,375 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:55:37,375 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:37,375 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:37,375 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:37,375 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. started... 2023-03-24 21:55:37,375 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. started... 2023-03-24 21:55:37,375 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 845f335cfe6009956f8aee7e54864e24 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-24 21:55:37,375 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d12810111afaf6de0915ac2e031e6b48 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-24 21:55:37,375 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing cc01e67bbfbc97fe3b57bdba3619e55a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-24 21:55:37,816 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324b8e8f9ee128e482e839e28bfeb0521b7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230324b8e8f9ee128e482e839e28bfeb0521b7 2023-03-24 21:55:37,816 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303247236da370b4f4f358649daaacbd17174 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303247236da370b4f4f358649daaacbd17174 2023-03-24 21:55:37,816 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324f167074a7c2244a6a5de07702f24642e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230324f167074a7c2244a6a5de07702f24642e 2023-03-24 21:55:37,832 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family1/1fb5b1c7cb384e5780995040ab8b9571 2023-03-24 21:55:37,832 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family1/c1203265e45047a68b9e70e61f1ea072 2023-03-24 21:55:37,865 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303241c5d25d3054640b984f55da09a4ae537 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303241c5d25d3054640b984f55da09a4ae537 2023-03-24 21:55:37,865 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324e27d49b8a673418f9f52c6770acde7fd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230324e27d49b8a673418f9f52c6770acde7fd 2023-03-24 21:55:38,231 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family1/0502f1a2a3414820b300f437039e3a2d 2023-03-24 21:55:38,269 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family2/e365b0180f764dbfa451d80b41aac3c3 2023-03-24 21:55:38,269 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family2/7856c844f0334be6a80adde07fa608e1 2023-03-24 21:55:38,272 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family1/1fb5b1c7cb384e5780995040ab8b9571 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/1fb5b1c7cb384e5780995040ab8b9571 2023-03-24 21:55:38,273 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family1/c1203265e45047a68b9e70e61f1ea072 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/c1203265e45047a68b9e70e61f1ea072 2023-03-24 21:55:38,296 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/c1203265e45047a68b9e70e61f1ea072, entries=4, sequenceid=9, filesize=5.5 K 2023-03-24 21:55:38,296 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/1fb5b1c7cb384e5780995040ab8b9571, entries=4, sequenceid=9, filesize=5.5 K 2023-03-24 21:55:38,296 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family2/7856c844f0334be6a80adde07fa608e1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/7856c844f0334be6a80adde07fa608e1 2023-03-24 21:55:38,296 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family2/e365b0180f764dbfa451d80b41aac3c3 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/e365b0180f764dbfa451d80b41aac3c3 2023-03-24 21:55:38,299 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/7856c844f0334be6a80adde07fa608e1, entries=2, sequenceid=9, filesize=5.1 K 2023-03-24 21:55:38,299 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/e365b0180f764dbfa451d80b41aac3c3, entries=2, sequenceid=9, filesize=5.1 K 2023-03-24 21:55:38,300 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for cc01e67bbfbc97fe3b57bdba3619e55a in 925ms, sequenceid=9, compaction requested=false 2023-03-24 21:55:38,300 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d12810111afaf6de0915ac2e031e6b48 in 925ms, sequenceid=9, compaction requested=false 2023-03-24 21:55:38,300 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for cc01e67bbfbc97fe3b57bdba3619e55a: 2023-03-24 21:55:38,300 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d12810111afaf6de0915ac2e031e6b48: 2023-03-24 21:55:38,300 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:38,300 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:38,300 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:55:38,300 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:55:38,663 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324374ba98a24a1486cb9b9451ced948b03 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230324374ba98a24a1486cb9b9451ced948b03 2023-03-24 21:55:39,068 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family2/0ac43fc372d04c24adf9c4bc81f0d93a 2023-03-24 21:55:39,071 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family1/0502f1a2a3414820b300f437039e3a2d as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/0502f1a2a3414820b300f437039e3a2d 2023-03-24 21:55:39,095 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/0502f1a2a3414820b300f437039e3a2d, entries=4, sequenceid=9, filesize=5.5 K 2023-03-24 21:55:39,095 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family2/0ac43fc372d04c24adf9c4bc81f0d93a as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/0ac43fc372d04c24adf9c4bc81f0d93a 2023-03-24 21:55:39,099 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/0ac43fc372d04c24adf9c4bc81f0d93a, entries=2, sequenceid=9, filesize=5.1 K 2023-03-24 21:55:39,099 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 845f335cfe6009956f8aee7e54864e24 in 1724ms, sequenceid=9, compaction requested=false 2023-03-24 21:55:39,100 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 845f335cfe6009956f8aee7e54864e24: 2023-03-24 21:55:39,100 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:39,100 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:55:39,100 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:55:39,100 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,100 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-24 21:55:39,100 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-24 21:55:39,101 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:39,101 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:39,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:39,101 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:39,101 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:39,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-24 21:55:39,103 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@139a4063[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:39,103 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-24 21:55:39,103 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:39,104 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:39,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:39,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:39,104 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-24 21:55:39,104 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,104 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:39,104 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-24 21:55:39,104 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:39,105 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,105 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-24 21:55:39,105 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:39,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:39,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:39,105 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-24 21:55:39,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:39,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:39,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:39,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:39,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:39,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:39,107 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-24 21:55:39,107 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-24 21:55:39,107 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:39,107 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:39,107 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-24 21:55:39,107 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:39,109 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:39,109 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:39,109 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:39,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:39,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:39,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:39,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:39,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:39,110 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:39,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:39,111 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:39,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:39,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,113 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,113 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:39,113 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:39,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:39,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-24 21:55:39,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:39,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-24 21:55:39,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:39,113 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:39,113 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:39,113 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:39,113 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:39,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:39,114 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:39,114 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:39,114 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:39,114 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:39,114 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:39,214 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:39,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:39,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:39,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:39,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:39,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:39,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:39,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-24 21:55:39,220 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-24 21:55:39,220 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:39,220 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-24 21:55:39,220 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-24 21:55:39,220 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:39,221 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:39,222 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,222 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:39,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:39,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:39,222 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:39,222 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:39,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:39,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:39,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:39,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:39,223 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-24 21:55:39,223 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-24 21:55:39,223 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-24 21:55:39,223 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-24 21:55:39,223 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:39,223 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-24 21:55:39,223 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:39,223 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:55:39,224 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:39,223 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:39,223 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. started... 2023-03-24 21:55:39,224 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. started... 2023-03-24 21:55:39,224 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:39,224 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. started... 2023-03-24 21:55:39,224 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 845f335cfe6009956f8aee7e54864e24 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-24 21:55:39,224 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d12810111afaf6de0915ac2e031e6b48 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-24 21:55:39,224 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing cc01e67bbfbc97fe3b57bdba3619e55a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-24 21:55:39,280 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324b81dcb7f8c5d4e10839268de2b3477fb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230324b81dcb7f8c5d4e10839268de2b3477fb 2023-03-24 21:55:39,280 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032499b5c6c888ad464fb4b6574d78fb0b6f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032499b5c6c888ad464fb4b6574d78fb0b6f 2023-03-24 21:55:39,668 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324721f8b69893f45c0b73511061d0498d0 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230324721f8b69893f45c0b73511061d0498d0 2023-03-24 21:55:39,697 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family1/f2216798c8934ea9a71edb752d19ce01 2023-03-24 21:55:39,697 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family1/7d1a2b5db02f48d7880a6aa0fa456ab6 2023-03-24 21:55:40,083 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family1/d3a75436e3ee4ec68117b3142c0e0a8c 2023-03-24 21:55:40,141 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032463e4c91cb96e47079f676d4410326639 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023032463e4c91cb96e47079f676d4410326639 2023-03-24 21:55:40,141 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303244436f9460c864f9e9c138a670118a989 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303244436f9460c864f9e9c138a670118a989 2023-03-24 21:55:40,151 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family2/c0e88c00e0894bc289d1fb1bd7ac46db 2023-03-24 21:55:40,152 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family2/f4b884ef750b4f52a5d9bf044f71d880 2023-03-24 21:55:40,158 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family1/f2216798c8934ea9a71edb752d19ce01 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/f2216798c8934ea9a71edb752d19ce01 2023-03-24 21:55:40,158 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family1/7d1a2b5db02f48d7880a6aa0fa456ab6 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/7d1a2b5db02f48d7880a6aa0fa456ab6 2023-03-24 21:55:40,182 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/7d1a2b5db02f48d7880a6aa0fa456ab6, entries=4, sequenceid=13, filesize=5.5 K 2023-03-24 21:55:40,182 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family2/c0e88c00e0894bc289d1fb1bd7ac46db as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/c0e88c00e0894bc289d1fb1bd7ac46db 2023-03-24 21:55:40,186 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/c0e88c00e0894bc289d1fb1bd7ac46db, entries=2, sequenceid=13, filesize=5.1 K 2023-03-24 21:55:40,187 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d12810111afaf6de0915ac2e031e6b48 in 963ms, sequenceid=13, compaction requested=true 2023-03-24 21:55:40,187 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d12810111afaf6de0915ac2e031e6b48: 2023-03-24 21:55:40,187 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:40,187 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:55:40,190 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/f2216798c8934ea9a71edb752d19ce01, entries=4, sequenceid=13, filesize=5.5 K 2023-03-24 21:55:40,191 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family2/f4b884ef750b4f52a5d9bf044f71d880 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/f4b884ef750b4f52a5d9bf044f71d880 2023-03-24 21:55:40,194 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/f4b884ef750b4f52a5d9bf044f71d880, entries=2, sequenceid=13, filesize=5.1 K 2023-03-24 21:55:40,195 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for cc01e67bbfbc97fe3b57bdba3619e55a in 971ms, sequenceid=13, compaction requested=true 2023-03-24 21:55:40,195 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for cc01e67bbfbc97fe3b57bdba3619e55a: 2023-03-24 21:55:40,195 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:40,195 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:55:40,520 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324e3c265fb72024c49aaf689081a973fef to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230324e3c265fb72024c49aaf689081a973fef 2023-03-24 21:55:40,528 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family2/acf4bccccf9141d2bd2556706b0d7c9d 2023-03-24 21:55:40,532 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family1/d3a75436e3ee4ec68117b3142c0e0a8c as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/d3a75436e3ee4ec68117b3142c0e0a8c 2023-03-24 21:55:40,561 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/d3a75436e3ee4ec68117b3142c0e0a8c, entries=4, sequenceid=13, filesize=5.5 K 2023-03-24 21:55:40,562 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family2/acf4bccccf9141d2bd2556706b0d7c9d as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/acf4bccccf9141d2bd2556706b0d7c9d 2023-03-24 21:55:40,566 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/acf4bccccf9141d2bd2556706b0d7c9d, entries=2, sequenceid=13, filesize=5.1 K 2023-03-24 21:55:40,567 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 845f335cfe6009956f8aee7e54864e24 in 1343ms, sequenceid=13, compaction requested=true 2023-03-24 21:55:40,567 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 845f335cfe6009956f8aee7e54864e24: 2023-03-24 21:55:40,567 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:40,567 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:55:40,567 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:55:40,567 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,567 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-24 21:55:40,567 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-24 21:55:40,570 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,570 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:40,571 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:40,571 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:40,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:40,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:40,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:40,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:40,572 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-24 21:55:40,572 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47d218d4[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:40,572 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-24 21:55:40,572 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:40,574 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:40,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:40,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:40,574 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-24 21:55:40,574 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-24 21:55:40,574 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,574 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:40,574 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:40,575 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:40,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:40,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:40,576 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-24 21:55:40,576 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:40,576 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-24 21:55:40,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:40,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:40,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:40,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:40,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:40,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-24 21:55:40,578 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-24 21:55:40,578 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:40,578 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:40,578 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-24 21:55:40,578 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:40,580 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:40,580 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:40,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:40,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:40,580 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:40,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:40,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:40,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:40,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:40,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:40,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:40,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:40,581 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:40,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:40,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,582 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:40,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:40,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:40,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,585 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:40,585 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,585 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:40,585 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:40,585 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:40,585 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,585 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:40,585 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:40,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-24 21:55:40,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:40,585 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:40,586 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:40,586 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:40,586 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:40,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:40,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:40,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:40,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:40,686 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:40,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:40,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:40,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:40,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. with WAL disabled. Data may be lost in the event of a crash. 2023-03-24 21:55:40,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:40,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:40,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-24 21:55:40,692 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-24 21:55:40,692 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:40,692 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-24 21:55:40,692 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-24 21:55:40,693 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:40,693 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:40,697 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:40,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:40,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:40,697 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:40,698 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:40,698 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:40,698 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:40,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:40,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:40,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-24 21:55:40,698 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-24 21:55:40,699 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-24 21:55:40,699 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-24 21:55:40,699 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:40,699 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-24 21:55:40,699 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:40,699 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. started... 2023-03-24 21:55:40,699 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:55:40,699 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:40,699 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 845f335cfe6009956f8aee7e54864e24 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-24 21:55:40,699 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:40,699 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:40,699 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. started... 2023-03-24 21:55:40,699 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. started... 2023-03-24 21:55:40,700 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d12810111afaf6de0915ac2e031e6b48 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-24 21:55:40,700 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing cc01e67bbfbc97fe3b57bdba3619e55a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-24 21:55:40,716 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-24 21:55:41,165 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324aedac4ba5cb047039f9c428e99b5c04f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230324aedac4ba5cb047039f9c428e99b5c04f 2023-03-24 21:55:41,170 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324042e4fbe8cb541cfb94b0ba50ace3ae1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230324042e4fbe8cb541cfb94b0ba50ace3ae1 2023-03-24 21:55:41,189 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303246cc177cb739043f087d5bcd46cbbfee8 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303246cc177cb739043f087d5bcd46cbbfee8 2023-03-24 21:55:41,196 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family1/3b802efeab85453e8b64a8ff0f05cb60 2023-03-24 21:55:41,239 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230324165adf1b9ff0474bb4c5f7c4c200b1a4 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230324165adf1b9ff0474bb4c5f7c4c200b1a4 2023-03-24 21:55:41,589 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family1/4806765c05a942c080559d807f18d432 2023-03-24 21:55:41,611 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family1/9d97c473b6e441b9a0a6cbaed6535623 2023-03-24 21:55:41,644 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family2/7005a3aac64c49a9aeb48097abbd6d6d 2023-03-24 21:55:41,651 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family1/3b802efeab85453e8b64a8ff0f05cb60 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/3b802efeab85453e8b64a8ff0f05cb60 2023-03-24 21:55:41,694 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/3b802efeab85453e8b64a8ff0f05cb60, entries=4, sequenceid=17, filesize=5.5 K 2023-03-24 21:55:41,695 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family2/7005a3aac64c49a9aeb48097abbd6d6d as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/7005a3aac64c49a9aeb48097abbd6d6d 2023-03-24 21:55:41,696 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230324ff4ac6d743554b0d8401359d8ed1018a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230324ff4ac6d743554b0d8401359d8ed1018a 2023-03-24 21:55:41,705 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/7005a3aac64c49a9aeb48097abbd6d6d, entries=2, sequenceid=17, filesize=5.1 K 2023-03-24 21:55:41,706 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 845f335cfe6009956f8aee7e54864e24 in 1007ms, sequenceid=17, compaction requested=true 2023-03-24 21:55:41,706 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 845f335cfe6009956f8aee7e54864e24: 2023-03-24 21:55:41,706 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:41,706 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:55:41,722 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324a7afff30535641f3a0a57ac69c83dd75 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230324a7afff30535641f3a0a57ac69c83dd75 2023-03-24 21:55:42,113 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family2/9e47ccaf4b0040269fd131bd10704180 2023-03-24 21:55:42,118 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family1/4806765c05a942c080559d807f18d432 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/4806765c05a942c080559d807f18d432 2023-03-24 21:55:42,137 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family2/cd059dc4e1d44eb79acaa3d00e58f3b7 2023-03-24 21:55:42,145 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family1/9d97c473b6e441b9a0a6cbaed6535623 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/9d97c473b6e441b9a0a6cbaed6535623 2023-03-24 21:55:42,154 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/4806765c05a942c080559d807f18d432, entries=4, sequenceid=17, filesize=5.5 K 2023-03-24 21:55:42,156 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family2/9e47ccaf4b0040269fd131bd10704180 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/9e47ccaf4b0040269fd131bd10704180 2023-03-24 21:55:42,166 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/9e47ccaf4b0040269fd131bd10704180, entries=2, sequenceid=17, filesize=5.1 K 2023-03-24 21:55:42,167 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d12810111afaf6de0915ac2e031e6b48 in 1468ms, sequenceid=17, compaction requested=true 2023-03-24 21:55:42,167 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d12810111afaf6de0915ac2e031e6b48: 2023-03-24 21:55:42,167 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:42,168 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:55:42,184 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/9d97c473b6e441b9a0a6cbaed6535623, entries=4, sequenceid=17, filesize=5.5 K 2023-03-24 21:55:42,185 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family2/cd059dc4e1d44eb79acaa3d00e58f3b7 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/cd059dc4e1d44eb79acaa3d00e58f3b7 2023-03-24 21:55:42,191 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/cd059dc4e1d44eb79acaa3d00e58f3b7, entries=2, sequenceid=17, filesize=5.1 K 2023-03-24 21:55:42,192 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for cc01e67bbfbc97fe3b57bdba3619e55a in 1492ms, sequenceid=17, compaction requested=true 2023-03-24 21:55:42,192 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for cc01e67bbfbc97fe3b57bdba3619e55a: 2023-03-24 21:55:42,192 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:42,192 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:55:42,192 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:55:42,192 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,192 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-24 21:55:42,192 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-24 21:55:42,194 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,195 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:42,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:42,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:42,195 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:42,195 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:42,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:42,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:42,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:42,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:42,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-24 21:55:42,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79bee758[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:42,198 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-24 21:55:42,198 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:42,199 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:42,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:42,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:42,199 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-24 21:55:42,199 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-24 21:55:42,199 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:42,199 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,199 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:42,202 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,202 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-24 21:55:42,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:42,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:42,202 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:42,202 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-24 21:55:42,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:42,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:42,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:42,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:42,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:42,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:42,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-24 21:55:42,206 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-24 21:55:42,206 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:42,206 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:42,206 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-24 21:55:42,206 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:42,208 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:42,208 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:42,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:42,208 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:42,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:42,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:42,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:42,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:42,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:42,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:42,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:42,209 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:42,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:42,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:42,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:42,211 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:42,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,215 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:42,215 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:42,215 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:42,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:42,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:42,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-24 21:55:42,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:42,215 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-24 21:55:42,215 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,216 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:42,216 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:42,216 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:42,216 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:42,216 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:42,216 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:42,216 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:42,216 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:42,316 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:42,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:42,329 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:42,338 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:42,348 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:42,350 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:42,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-03-24 21:55:42,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-24 21:55:42,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-24 21:55:42,458 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-24 21:55:42,458 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:42,458 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-24 21:55:42,458 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-24 21:55:42,459 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:42,459 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:42,462 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:42,462 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:42,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:42,462 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:42,462 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-24 21:55:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:42,463 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-24 21:55:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:42,464 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-24 21:55:42,464 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-24 21:55:42,464 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-24 21:55:42,464 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-24 21:55:42,464 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-24 21:55:42,464 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-24 21:55:42,467 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:42,467 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. started... 2023-03-24 21:55:42,467 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:42,467 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. started... 2023-03-24 21:55:42,467 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 845f335cfe6009956f8aee7e54864e24 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-24 21:55:42,467 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing cc01e67bbfbc97fe3b57bdba3619e55a 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-24 21:55:42,467 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-24 21:55:42,467 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-24 21:55:42,467 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:42,467 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. started... 2023-03-24 21:55:42,468 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d12810111afaf6de0915ac2e031e6b48 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-24 21:55:42,949 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family1/51cbff30d9c44bfb9a70f62cdf6f5f69 2023-03-24 21:55:42,952 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family1/5df6a7bdd8214c76abcb4c430fdb372f 2023-03-24 21:55:42,961 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family1/25430910c4b942798c7e5a3cf78211cf 2023-03-24 21:55:42,983 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51cbff30d9c44bfb9a70f62cdf6f5f69 2023-03-24 21:55:42,988 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5df6a7bdd8214c76abcb4c430fdb372f 2023-03-24 21:55:42,999 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25430910c4b942798c7e5a3cf78211cf 2023-03-24 21:55:43,424 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family2/1f54cb0eeda6440493d50c84e2a02c76 2023-03-24 21:55:43,429 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f54cb0eeda6440493d50c84e2a02c76 2023-03-24 21:55:43,431 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family2/80db7525da7a4b2e868d9a8c8f91ed38 2023-03-24 21:55:43,431 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family1/51cbff30d9c44bfb9a70f62cdf6f5f69 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/51cbff30d9c44bfb9a70f62cdf6f5f69 2023-03-24 21:55:43,434 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 80db7525da7a4b2e868d9a8c8f91ed38 2023-03-24 21:55:43,435 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family1/25430910c4b942798c7e5a3cf78211cf as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/25430910c4b942798c7e5a3cf78211cf 2023-03-24 21:55:43,463 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25430910c4b942798c7e5a3cf78211cf 2023-03-24 21:55:43,463 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/25430910c4b942798c7e5a3cf78211cf, entries=3, sequenceid=23, filesize=5.2 K 2023-03-24 21:55:43,464 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family2/80db7525da7a4b2e868d9a8c8f91ed38 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/80db7525da7a4b2e868d9a8c8f91ed38 2023-03-24 21:55:43,465 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51cbff30d9c44bfb9a70f62cdf6f5f69 2023-03-24 21:55:43,466 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/51cbff30d9c44bfb9a70f62cdf6f5f69, entries=3, sequenceid=23, filesize=5.2 K 2023-03-24 21:55:43,466 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family2/1f54cb0eeda6440493d50c84e2a02c76 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/1f54cb0eeda6440493d50c84e2a02c76 2023-03-24 21:55:43,468 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 80db7525da7a4b2e868d9a8c8f91ed38 2023-03-24 21:55:43,468 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/80db7525da7a4b2e868d9a8c8f91ed38, entries=1, sequenceid=23, filesize=5.0 K 2023-03-24 21:55:43,469 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for cc01e67bbfbc97fe3b57bdba3619e55a in 1002ms, sequenceid=23, compaction requested=true 2023-03-24 21:55:43,469 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for cc01e67bbfbc97fe3b57bdba3619e55a: 2023-03-24 21:55:43,469 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:43,469 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-24 21:55:43,470 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f54cb0eeda6440493d50c84e2a02c76 2023-03-24 21:55:43,470 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/1f54cb0eeda6440493d50c84e2a02c76, entries=1, sequenceid=23, filesize=5.0 K 2023-03-24 21:55:43,471 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 845f335cfe6009956f8aee7e54864e24 in 1004ms, sequenceid=23, compaction requested=true 2023-03-24 21:55:43,471 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 845f335cfe6009956f8aee7e54864e24: 2023-03-24 21:55:43,471 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:43,471 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-24 21:55:43,823 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family2/cea86de5e5c74ffb8fbaaa5a9c3b8ab1 2023-03-24 21:55:43,827 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cea86de5e5c74ffb8fbaaa5a9c3b8ab1 2023-03-24 21:55:43,827 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family1/5df6a7bdd8214c76abcb4c430fdb372f as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/5df6a7bdd8214c76abcb4c430fdb372f 2023-03-24 21:55:43,860 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5df6a7bdd8214c76abcb4c430fdb372f 2023-03-24 21:55:43,861 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/5df6a7bdd8214c76abcb4c430fdb372f, entries=3, sequenceid=23, filesize=5.2 K 2023-03-24 21:55:43,861 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family2/cea86de5e5c74ffb8fbaaa5a9c3b8ab1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/cea86de5e5c74ffb8fbaaa5a9c3b8ab1 2023-03-24 21:55:43,865 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cea86de5e5c74ffb8fbaaa5a9c3b8ab1 2023-03-24 21:55:43,866 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/cea86de5e5c74ffb8fbaaa5a9c3b8ab1, entries=1, sequenceid=23, filesize=5.0 K 2023-03-24 21:55:43,866 INFO [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d12810111afaf6de0915ac2e031e6b48 in 1398ms, sequenceid=23, compaction requested=true 2023-03-24 21:55:43,867 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d12810111afaf6de0915ac2e031e6b48: 2023-03-24 21:55:43,867 DEBUG [rs(jenkins-hbase9.apache.org,41807,1679694819103)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:43,867 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-24 21:55:43,867 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-24 21:55:43,867 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,867 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-24 21:55:43,867 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-24 21:55:43,868 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,868 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:43,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:43,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:43,869 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:43,869 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-24 21:55:43,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:43,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:43,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:43,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:43,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41807,1679694819103' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-24 21:55:43,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f53649f[Count = 0] remaining members to acquire global barrier 2023-03-24 21:55:43,870 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-24 21:55:43,870 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:43,872 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:43,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:43,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:43,872 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-24 21:55:43,872 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,872 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-24 21:55:43,872 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-24 21:55:43,873 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase9.apache.org,41807,1679694819103' in zk 2023-03-24 21:55:43,874 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,874 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-24 21:55:43,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:43,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:43,874 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:43,874 DEBUG [member: 'jenkins-hbase9.apache.org,41807,1679694819103' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-24 21:55:43,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:43,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:43,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:43,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:43,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:43,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase9.apache.org,41807,1679694819103': 2023-03-24 21:55:43,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41807,1679694819103' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-24 21:55:43,876 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-24 21:55:43,877 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-24 21:55:43,877 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-24 21:55:43,877 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-24 21:55:43,877 INFO [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-24 21:55:43,878 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:43,878 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:43,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:43,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:43,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-24 21:55:43,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-24 21:55:43,878 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:43,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:43,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-24 21:55:43,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:43,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:43,879 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:43,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:43,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-24 21:55:43,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:43,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,880 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-24 21:55:43,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-24 21:55:43,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,882 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-24 21:55:43,882 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-24 21:55:43,882 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:43,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-24 21:55:43,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-24 21:55:43,882 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-24 21:55:43,882 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-24 21:55:43,882 DEBUG [(jenkins-hbase9.apache.org,46449,1679694817811)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-24 21:55:43,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-24 21:55:43,883 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:43,883 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-24 21:55:43,883 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-24 21:55:43,883 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:43,883 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-24 21:55:43,883 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-24 21:55:43,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-24 21:55:43,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-24 21:55:43,983 DEBUG [Listener at localhost/39907] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-24 21:55:43,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46449] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-24 21:55:43,985 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:43,985 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): 845f335cfe6009956f8aee7e54864e24/family1 is initiating major compaction (all files) 2023-03-24 21:55:43,985 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:43,985 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:43,985 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of 845f335cfe6009956f8aee7e54864e24/family1 in testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:43,985 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/affc9d0aad9242beb3c6d2070b928b7d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/0502f1a2a3414820b300f437039e3a2d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/d3a75436e3ee4ec68117b3142c0e0a8c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/3b802efeab85453e8b64a8ff0f05cb60, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/51cbff30d9c44bfb9a70f62cdf6f5f69] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp, totalSize=27.1 K 2023-03-24 21:55:43,986 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting affc9d0aad9242beb3c6d2070b928b7d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694935508 2023-03-24 21:55:43,986 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 0502f1a2a3414820b300f437039e3a2d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694937367 2023-03-24 21:55:43,987 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting d3a75436e3ee4ec68117b3142c0e0a8c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694939216 2023-03-24 21:55:43,987 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 3b802efeab85453e8b64a8ff0f05cb60, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694940688 2023-03-24 21:55:43,987 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 51cbff30d9c44bfb9a70f62cdf6f5f69, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-24 21:55:43,997 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): 845f335cfe6009956f8aee7e54864e24#family1#compaction#197 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:44,043 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303246323c5feef354ef8aec1f579cee77db6_del 2023-03-24 21:55:44,043 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303246323c5feef354ef8aec1f579cee77db6_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303246323c5feef354ef8aec1f579cee77db6_del 2023-03-24 21:55:44,099 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9b3706d39ece46c8a2be964d05c012ce 2023-03-24 21:55:44,100 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family1/9b3706d39ece46c8a2be964d05c012ce as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/9b3706d39ece46c8a2be964d05c012ce 2023-03-24 21:55:44,127 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9b3706d39ece46c8a2be964d05c012ce 2023-03-24 21:55:44,128 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 845f335cfe6009956f8aee7e54864e24/family1 of 845f335cfe6009956f8aee7e54864e24 into 9b3706d39ece46c8a2be964d05c012ce(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:44,129 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for 845f335cfe6009956f8aee7e54864e24: 2023-03-24 21:55:44,129 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:44,129 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): 845f335cfe6009956f8aee7e54864e24/family2 is initiating major compaction (all files) 2023-03-24 21:55:44,129 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:44,129 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:44,130 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of 845f335cfe6009956f8aee7e54864e24/family2 in testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:44,130 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/bc4bb3977d06473cbf5e84605b27363f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/0ac43fc372d04c24adf9c4bc81f0d93a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/acf4bccccf9141d2bd2556706b0d7c9d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/7005a3aac64c49a9aeb48097abbd6d6d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/1f54cb0eeda6440493d50c84e2a02c76] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp, totalSize=25.4 K 2023-03-24 21:55:44,131 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting bc4bb3977d06473cbf5e84605b27363f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694935508 2023-03-24 21:55:44,131 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 0ac43fc372d04c24adf9c4bc81f0d93a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694937367 2023-03-24 21:55:44,131 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting acf4bccccf9141d2bd2556706b0d7c9d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694939216 2023-03-24 21:55:44,132 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 7005a3aac64c49a9aeb48097abbd6d6d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694940688 2023-03-24 21:55:44,132 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 1f54cb0eeda6440493d50c84e2a02c76, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-24 21:55:44,141 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): 845f335cfe6009956f8aee7e54864e24#family2#compaction#198 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:44,148 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023032406ba0c3c9e1e4c9692b9db6b8f7322f2_del 2023-03-24 21:55:44,149 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032406ba0c3c9e1e4c9692b9db6b8f7322f2_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023032406ba0c3c9e1e4c9692b9db6b8f7322f2_del 2023-03-24 21:55:44,168 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da482f8df0ac470f91daedc29dd6174d 2023-03-24 21:55:44,169 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/.tmp/family2/da482f8df0ac470f91daedc29dd6174d as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/da482f8df0ac470f91daedc29dd6174d 2023-03-24 21:55:44,173 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da482f8df0ac470f91daedc29dd6174d 2023-03-24 21:55:44,174 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 845f335cfe6009956f8aee7e54864e24/family2 of 845f335cfe6009956f8aee7e54864e24 into da482f8df0ac470f91daedc29dd6174d(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:44,174 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for 845f335cfe6009956f8aee7e54864e24: 2023-03-24 21:55:44,174 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:44,175 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): cc01e67bbfbc97fe3b57bdba3619e55a/family1 is initiating major compaction (all files) 2023-03-24 21:55:44,175 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:44,175 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:44,176 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of cc01e67bbfbc97fe3b57bdba3619e55a/family1 in testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:44,176 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/e4ea0ea6d14246b0998af1050b4977a6, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/1fb5b1c7cb384e5780995040ab8b9571, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/f2216798c8934ea9a71edb752d19ce01, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/9d97c473b6e441b9a0a6cbaed6535623, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/25430910c4b942798c7e5a3cf78211cf] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp, totalSize=27.1 K 2023-03-24 21:55:44,177 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting e4ea0ea6d14246b0998af1050b4977a6, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694935507 2023-03-24 21:55:44,177 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 1fb5b1c7cb384e5780995040ab8b9571, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694937366 2023-03-24 21:55:44,177 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting f2216798c8934ea9a71edb752d19ce01, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694939216 2023-03-24 21:55:44,178 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 9d97c473b6e441b9a0a6cbaed6535623, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694940688 2023-03-24 21:55:44,178 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 25430910c4b942798c7e5a3cf78211cf, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-24 21:55:44,186 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): cc01e67bbfbc97fe3b57bdba3619e55a#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:44,195 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/affc9d0aad9242beb3c6d2070b928b7d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/0502f1a2a3414820b300f437039e3a2d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/d3a75436e3ee4ec68117b3142c0e0a8c, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/3b802efeab85453e8b64a8ff0f05cb60, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/51cbff30d9c44bfb9a70f62cdf6f5f69] to archive 2023-03-24 21:55:44,199 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:44,201 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/affc9d0aad9242beb3c6d2070b928b7d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/affc9d0aad9242beb3c6d2070b928b7d 2023-03-24 21:55:44,202 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/0502f1a2a3414820b300f437039e3a2d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/0502f1a2a3414820b300f437039e3a2d 2023-03-24 21:55:44,203 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/d3a75436e3ee4ec68117b3142c0e0a8c to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/d3a75436e3ee4ec68117b3142c0e0a8c 2023-03-24 21:55:44,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/3b802efeab85453e8b64a8ff0f05cb60 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/3b802efeab85453e8b64a8ff0f05cb60 2023-03-24 21:55:44,207 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/51cbff30d9c44bfb9a70f62cdf6f5f69 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/51cbff30d9c44bfb9a70f62cdf6f5f69 2023-03-24 21:55:44,208 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/bc4bb3977d06473cbf5e84605b27363f, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/0ac43fc372d04c24adf9c4bc81f0d93a, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/acf4bccccf9141d2bd2556706b0d7c9d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/7005a3aac64c49a9aeb48097abbd6d6d, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/1f54cb0eeda6440493d50c84e2a02c76] to archive 2023-03-24 21:55:44,209 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:44,210 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/bc4bb3977d06473cbf5e84605b27363f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/bc4bb3977d06473cbf5e84605b27363f 2023-03-24 21:55:44,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/0ac43fc372d04c24adf9c4bc81f0d93a to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/0ac43fc372d04c24adf9c4bc81f0d93a 2023-03-24 21:55:44,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/acf4bccccf9141d2bd2556706b0d7c9d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/acf4bccccf9141d2bd2556706b0d7c9d 2023-03-24 21:55:44,221 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/7005a3aac64c49a9aeb48097abbd6d6d to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/7005a3aac64c49a9aeb48097abbd6d6d 2023-03-24 21:55:44,222 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/1f54cb0eeda6440493d50c84e2a02c76 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family2/1f54cb0eeda6440493d50c84e2a02c76 2023-03-24 21:55:44,257 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230324c709061c76294512a1ca4c70781af3cc_del 2023-03-24 21:55:44,257 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324c709061c76294512a1ca4c70781af3cc_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230324c709061c76294512a1ca4c70781af3cc_del 2023-03-24 21:55:44,728 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5de5bb56452f419f8dcc2abc9713e301 2023-03-24 21:55:44,729 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family1/5de5bb56452f419f8dcc2abc9713e301 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/5de5bb56452f419f8dcc2abc9713e301 2023-03-24 21:55:44,758 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5de5bb56452f419f8dcc2abc9713e301 2023-03-24 21:55:44,759 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in cc01e67bbfbc97fe3b57bdba3619e55a/family1 of cc01e67bbfbc97fe3b57bdba3619e55a into 5de5bb56452f419f8dcc2abc9713e301(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:44,759 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for cc01e67bbfbc97fe3b57bdba3619e55a: 2023-03-24 21:55:44,759 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:44,760 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): cc01e67bbfbc97fe3b57bdba3619e55a/family2 is initiating major compaction (all files) 2023-03-24 21:55:44,760 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:44,760 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:44,760 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of cc01e67bbfbc97fe3b57bdba3619e55a/family2 in testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:44,760 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/075c172d14b94eb389a89945e38ec7cd, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/e365b0180f764dbfa451d80b41aac3c3, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/f4b884ef750b4f52a5d9bf044f71d880, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/cd059dc4e1d44eb79acaa3d00e58f3b7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/80db7525da7a4b2e868d9a8c8f91ed38] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp, totalSize=25.4 K 2023-03-24 21:55:44,761 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 075c172d14b94eb389a89945e38ec7cd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694935507 2023-03-24 21:55:44,761 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting e365b0180f764dbfa451d80b41aac3c3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694937366 2023-03-24 21:55:44,762 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting f4b884ef750b4f52a5d9bf044f71d880, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694939216 2023-03-24 21:55:44,762 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting cd059dc4e1d44eb79acaa3d00e58f3b7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694940688 2023-03-24 21:55:44,762 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 80db7525da7a4b2e868d9a8c8f91ed38, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-24 21:55:44,773 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): cc01e67bbfbc97fe3b57bdba3619e55a#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:44,790 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230324fb7cd089497b4040a4436d2e93fa2cec_del 2023-03-24 21:55:44,791 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324fb7cd089497b4040a4436d2e93fa2cec_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230324fb7cd089497b4040a4436d2e93fa2cec_del 2023-03-24 21:55:44,797 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/e4ea0ea6d14246b0998af1050b4977a6, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/1fb5b1c7cb384e5780995040ab8b9571, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/f2216798c8934ea9a71edb752d19ce01, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/9d97c473b6e441b9a0a6cbaed6535623, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/25430910c4b942798c7e5a3cf78211cf] to archive 2023-03-24 21:55:44,797 WARN [DataStreamer for file /user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324c4474a8b1aff49e5b8326693715b48a6] hdfs.DataStreamer(982): Caught exception java.lang.InterruptedException at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1257) at java.lang.Thread.join(Thread.java:1331) at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:980) at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:630) at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:807) 2023-03-24 21:55:44,798 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:44,800 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/e4ea0ea6d14246b0998af1050b4977a6 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/e4ea0ea6d14246b0998af1050b4977a6 2023-03-24 21:55:44,803 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/1fb5b1c7cb384e5780995040ab8b9571 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/1fb5b1c7cb384e5780995040ab8b9571 2023-03-24 21:55:44,805 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/f2216798c8934ea9a71edb752d19ce01 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/f2216798c8934ea9a71edb752d19ce01 2023-03-24 21:55:44,807 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/9d97c473b6e441b9a0a6cbaed6535623 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/9d97c473b6e441b9a0a6cbaed6535623 2023-03-24 21:55:44,809 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/25430910c4b942798c7e5a3cf78211cf to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/25430910c4b942798c7e5a3cf78211cf 2023-03-24 21:55:44,810 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d2cd979add704196aad32d7d9268d492 2023-03-24 21:55:44,812 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/.tmp/family2/d2cd979add704196aad32d7d9268d492 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/d2cd979add704196aad32d7d9268d492 2023-03-24 21:55:44,816 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d2cd979add704196aad32d7d9268d492 2023-03-24 21:55:44,817 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in cc01e67bbfbc97fe3b57bdba3619e55a/family2 of cc01e67bbfbc97fe3b57bdba3619e55a into d2cd979add704196aad32d7d9268d492(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:44,817 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for cc01e67bbfbc97fe3b57bdba3619e55a: 2023-03-24 21:55:44,817 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:44,817 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): d12810111afaf6de0915ac2e031e6b48/family1 is initiating major compaction (all files) 2023-03-24 21:55:44,817 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:44,818 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:44,819 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of d12810111afaf6de0915ac2e031e6b48/family1 in testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:44,819 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/b4632e0ea5594ea0ae55575cf604ed56, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/c1203265e45047a68b9e70e61f1ea072, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/7d1a2b5db02f48d7880a6aa0fa456ab6, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/4806765c05a942c080559d807f18d432, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/5df6a7bdd8214c76abcb4c430fdb372f] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp, totalSize=27.1 K 2023-03-24 21:55:44,819 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting b4632e0ea5594ea0ae55575cf604ed56, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694935508 2023-03-24 21:55:44,820 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting c1203265e45047a68b9e70e61f1ea072, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694937366 2023-03-24 21:55:44,820 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 7d1a2b5db02f48d7880a6aa0fa456ab6, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694939216 2023-03-24 21:55:44,823 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 4806765c05a942c080559d807f18d432, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694940688 2023-03-24 21:55:44,823 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 5df6a7bdd8214c76abcb4c430fdb372f, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-24 21:55:44,843 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): d12810111afaf6de0915ac2e031e6b48#family1#compaction#201 average throughput is 0.14 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:44,895 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/075c172d14b94eb389a89945e38ec7cd, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/e365b0180f764dbfa451d80b41aac3c3, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/f4b884ef750b4f52a5d9bf044f71d880, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/cd059dc4e1d44eb79acaa3d00e58f3b7, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/80db7525da7a4b2e868d9a8c8f91ed38] to archive 2023-03-24 21:55:44,896 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:44,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/075c172d14b94eb389a89945e38ec7cd to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/075c172d14b94eb389a89945e38ec7cd 2023-03-24 21:55:44,899 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/e365b0180f764dbfa451d80b41aac3c3 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/e365b0180f764dbfa451d80b41aac3c3 2023-03-24 21:55:44,900 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/f4b884ef750b4f52a5d9bf044f71d880 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/f4b884ef750b4f52a5d9bf044f71d880 2023-03-24 21:55:44,901 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/cd059dc4e1d44eb79acaa3d00e58f3b7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/cd059dc4e1d44eb79acaa3d00e58f3b7 2023-03-24 21:55:44,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/80db7525da7a4b2e868d9a8c8f91ed38 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family2/80db7525da7a4b2e868d9a8c8f91ed38 2023-03-24 21:55:45,291 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023032417e28ab03198401c9efb790d54175546_del 2023-03-24 21:55:45,291 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032417e28ab03198401c9efb790d54175546_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032417e28ab03198401c9efb790d54175546_del 2023-03-24 21:55:45,758 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d39c9b6759514720956028dfd03a5f02 2023-03-24 21:55:45,759 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family1/d39c9b6759514720956028dfd03a5f02 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/d39c9b6759514720956028dfd03a5f02 2023-03-24 21:55:45,791 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d39c9b6759514720956028dfd03a5f02 2023-03-24 21:55:45,792 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d12810111afaf6de0915ac2e031e6b48/family1 of d12810111afaf6de0915ac2e031e6b48 into d39c9b6759514720956028dfd03a5f02(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:45,792 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for d12810111afaf6de0915ac2e031e6b48: 2023-03-24 21:55:45,792 DEBUG [Listener at localhost/39907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-24 21:55:45,792 DEBUG [Listener at localhost/39907] regionserver.HStore(1912): d12810111afaf6de0915ac2e031e6b48/family2 is initiating major compaction (all files) 2023-03-24 21:55:45,792 INFO [Listener at localhost/39907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-24 21:55:45,792 INFO [Listener at localhost/39907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-24 21:55:45,793 INFO [Listener at localhost/39907] regionserver.HRegion(2259): Starting compaction of d12810111afaf6de0915ac2e031e6b48/family2 in testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:45,793 INFO [Listener at localhost/39907] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/38311ce68cb4477eb01f6a2d78fdaf12, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/7856c844f0334be6a80adde07fa608e1, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/c0e88c00e0894bc289d1fb1bd7ac46db, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/9e47ccaf4b0040269fd131bd10704180, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/cea86de5e5c74ffb8fbaaa5a9c3b8ab1] into tmpdir=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp, totalSize=25.4 K 2023-03-24 21:55:45,793 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 38311ce68cb4477eb01f6a2d78fdaf12, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679694935508 2023-03-24 21:55:45,794 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 7856c844f0334be6a80adde07fa608e1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679694937366 2023-03-24 21:55:45,795 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting c0e88c00e0894bc289d1fb1bd7ac46db, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679694939216 2023-03-24 21:55:45,795 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/b4632e0ea5594ea0ae55575cf604ed56, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/c1203265e45047a68b9e70e61f1ea072, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/7d1a2b5db02f48d7880a6aa0fa456ab6, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/4806765c05a942c080559d807f18d432, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/5df6a7bdd8214c76abcb4c430fdb372f] to archive 2023-03-24 21:55:45,795 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting 9e47ccaf4b0040269fd131bd10704180, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679694940688 2023-03-24 21:55:45,796 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:45,796 DEBUG [Listener at localhost/39907] compactions.Compactor(207): Compacting cea86de5e5c74ffb8fbaaa5a9c3b8ab1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-24 21:55:45,798 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/b4632e0ea5594ea0ae55575cf604ed56 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/b4632e0ea5594ea0ae55575cf604ed56 2023-03-24 21:55:45,803 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/c1203265e45047a68b9e70e61f1ea072 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/c1203265e45047a68b9e70e61f1ea072 2023-03-24 21:55:45,804 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/7d1a2b5db02f48d7880a6aa0fa456ab6 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/7d1a2b5db02f48d7880a6aa0fa456ab6 2023-03-24 21:55:45,806 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/4806765c05a942c080559d807f18d432 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/4806765c05a942c080559d807f18d432 2023-03-24 21:55:45,809 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/5df6a7bdd8214c76abcb4c430fdb372f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/5df6a7bdd8214c76abcb4c430fdb372f 2023-03-24 21:55:45,812 INFO [Listener at localhost/39907] throttle.PressureAwareThroughputController(145): d12810111afaf6de0915ac2e031e6b48#family2#compaction#202 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-24 21:55:45,822 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023032414bf7830e65d43f186b18d4ad713b4b2_del 2023-03-24 21:55:45,822 INFO [Listener at localhost/39907] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032414bf7830e65d43f186b18d4ad713b4b2_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023032414bf7830e65d43f186b18d4ad713b4b2_del 2023-03-24 21:55:45,834 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for afd9c1fa72374ccda33c7294664bb60a 2023-03-24 21:55:45,835 DEBUG [Listener at localhost/39907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/.tmp/family2/afd9c1fa72374ccda33c7294664bb60a as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/afd9c1fa72374ccda33c7294664bb60a 2023-03-24 21:55:45,838 INFO [Listener at localhost/39907] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for afd9c1fa72374ccda33c7294664bb60a 2023-03-24 21:55:45,839 INFO [Listener at localhost/39907] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d12810111afaf6de0915ac2e031e6b48/family2 of d12810111afaf6de0915ac2e031e6b48 into afd9c1fa72374ccda33c7294664bb60a(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-24 21:55:45,839 DEBUG [Listener at localhost/39907] regionserver.HRegion(2289): Compaction status journal for d12810111afaf6de0915ac2e031e6b48: 2023-03-24 21:55:45,843 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:45,846 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:45,849 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:45,851 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:45,857 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46449] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCFV\x05\xAC6\xDB:\xEB\xDA\xBE\xF0P\xA8\xF6p\xB4"\x10\x86\xB8S\xB9kg)\x11\xEE>\x01T\xB7Ns\x80*\x10\x13\xC8\xD3\x04\xBD$X\xB6\xC2V\x1EN\xC9|2\x132\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-03-24 21:55:45,881 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-24 21:55:45,882 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-24 21:55:45,883 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:55:45,883 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679695545883 2023-03-24 21:55:45,884 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-24 21:55:45,895 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/38311ce68cb4477eb01f6a2d78fdaf12, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/7856c844f0334be6a80adde07fa608e1, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/c0e88c00e0894bc289d1fb1bd7ac46db, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/9e47ccaf4b0040269fd131bd10704180, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/cea86de5e5c74ffb8fbaaa5a9c3b8ab1] to archive 2023-03-24 21:55:45,896 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:45,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/38311ce68cb4477eb01f6a2d78fdaf12 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/38311ce68cb4477eb01f6a2d78fdaf12 2023-03-24 21:55:45,899 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/7856c844f0334be6a80adde07fa608e1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/7856c844f0334be6a80adde07fa608e1 2023-03-24 21:55:45,901 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/c0e88c00e0894bc289d1fb1bd7ac46db to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/c0e88c00e0894bc289d1fb1bd7ac46db 2023-03-24 21:55:45,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/9e47ccaf4b0040269fd131bd10704180 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/9e47ccaf4b0040269fd131bd10704180 2023-03-24 21:55:45,903 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/cea86de5e5c74ffb8fbaaa5a9c3b8ab1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family2/cea86de5e5c74ffb8fbaaa5a9c3b8ab1 2023-03-24 21:55:46,318 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-03-24 21:55:46,359 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230324c709061c76294512a1ca4c70781af3cc_del 2023-03-24 21:55:46,385 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023032417e28ab03198401c9efb790d54175546_del 2023-03-24 21:55:46,417 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303246323c5feef354ef8aec1f579cee77db6_del 2023-03-24 21:55:46,417 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-24 21:55:46,417 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] zookeeper.ReadOnlyZKClient(139): Connect 0x4f0c350d to 127.0.0.1:54680 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-24 21:55:46,423 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@27527d54, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-24 21:55:46,423 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230324 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-24 21:55:46,592 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230324262e4d1690494ef9b149e18f11d2335f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230324262e4d1690494ef9b149e18f11d2335f 2023-03-24 21:55:46,593 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230324 table='testMajorCompactionFromAdmin' column='family1' 2023-03-24 21:55:46,606 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-24 21:55:46,609 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:50140, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-24 21:55:46,628 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-24 21:55:46,630 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:50004, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-24 21:55:46,669 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230324/family1/e304a541f26144749f88bacb7399868e first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-24 21:55:46,670 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230324/family1/e304a541f26144749f88bacb7399868e}] 2023-03-24 21:55:46,675 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230324/family1/e304a541f26144749f88bacb7399868e for inclusion in cc01e67bbfbc97fe3b57bdba3619e55a/family1 2023-03-24 21:55:46,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-24 21:55:46,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first= last=1 2023-03-24 21:55:46,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for cc01e67bbfbc97fe3b57bdba3619e55a: 2023-03-24 21:55:46,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230324/family1/e304a541f26144749f88bacb7399868e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__285rst12d09er8roqrng02qam6d6l1366eg3o5bo9l4cns63gc5avp7mq9mpbeub/family1/e304a541f26144749f88bacb7399868e 2023-03-24 21:55:46,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__285rst12d09er8roqrng02qam6d6l1366eg3o5bo9l4cns63gc5avp7mq9mpbeub/family1/e304a541f26144749f88bacb7399868e as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/6dfab07bc361477b94974b33a07b293e_SeqId_28_ 2023-03-24 21:55:46,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__285rst12d09er8roqrng02qam6d6l1366eg3o5bo9l4cns63gc5avp7mq9mpbeub/family1/e304a541f26144749f88bacb7399868e into cc01e67bbfbc97fe3b57bdba3619e55a/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/6dfab07bc361477b94974b33a07b293e_SeqId_28_ - updating store file list. 2023-03-24 21:55:46,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/6dfab07bc361477b94974b33a07b293e_SeqId_28_ into cc01e67bbfbc97fe3b57bdba3619e55a/family1 2023-03-24 21:55:46,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__285rst12d09er8roqrng02qam6d6l1366eg3o5bo9l4cns63gc5avp7mq9mpbeub/family1/e304a541f26144749f88bacb7399868e into cc01e67bbfbc97fe3b57bdba3619e55a/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/family1/6dfab07bc361477b94974b33a07b293e_SeqId_28_) 2023-03-24 21:55:46,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__285rst12d09er8roqrng02qam6d6l1366eg3o5bo9l4cns63gc5avp7mq9mpbeub/family1/e304a541f26144749f88bacb7399868e 2023-03-24 21:55:46,726 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230324 table='testMajorCompactionFromAdmin' column='family1' 2023-03-24 21:55:46,726 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:46,728 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303246cc177cb739043f087d5bcd46cbbfee8 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303246cc177cb739043f087d5bcd46cbbfee8 2023-03-24 21:55:46,729 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303246d8dcbdeae9b498c890bb6632dc53936 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303246d8dcbdeae9b498c890bb6632dc53936 2023-03-24 21:55:46,731 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303247236da370b4f4f358649daaacbd17174 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303247236da370b4f4f358649daaacbd17174 2023-03-24 21:55:46,732 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230324b81dcb7f8c5d4e10839268de2b3477fb to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230324b81dcb7f8c5d4e10839268de2b3477fb 2023-03-24 21:55:46,732 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-24 21:55:46,732 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230324' 2023-03-24 21:55:46,732 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230324 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-24 21:55:46,732 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230324 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-24 21:55:46,911 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303244010a9277ea44be3bc44cd6aa69b9168 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303244010a9277ea44be3bc44cd6aa69b9168 2023-03-24 21:55:46,911 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230324 table='testMajorCompactionFromAdmin' column='family1' 2023-03-24 21:55:46,960 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230324/family1/f813a4a0e0c64ff88a70398538f1daff first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-24 21:55:46,961 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230324/family1/f813a4a0e0c64ff88a70398538f1daff}] 2023-03-24 21:55:46,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230324/family1/f813a4a0e0c64ff88a70398538f1daff for inclusion in d12810111afaf6de0915ac2e031e6b48/family1 2023-03-24 21:55:46,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-24 21:55:46,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-24 21:55:46,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for d12810111afaf6de0915ac2e031e6b48: 2023-03-24 21:55:46,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230324/family1/f813a4a0e0c64ff88a70398538f1daff to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__c786nvq22e861n2s1k8i0f28h2ulqhvpg45db7iua25600n0dn9886n8c389eq2p/family1/f813a4a0e0c64ff88a70398538f1daff 2023-03-24 21:55:46,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__c786nvq22e861n2s1k8i0f28h2ulqhvpg45db7iua25600n0dn9886n8c389eq2p/family1/f813a4a0e0c64ff88a70398538f1daff as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/ced9dab4c039457e94224861eab85b5b_SeqId_28_ 2023-03-24 21:55:46,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__c786nvq22e861n2s1k8i0f28h2ulqhvpg45db7iua25600n0dn9886n8c389eq2p/family1/f813a4a0e0c64ff88a70398538f1daff into d12810111afaf6de0915ac2e031e6b48/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/ced9dab4c039457e94224861eab85b5b_SeqId_28_ - updating store file list. 2023-03-24 21:55:47,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/ced9dab4c039457e94224861eab85b5b_SeqId_28_ into d12810111afaf6de0915ac2e031e6b48/family1 2023-03-24 21:55:47,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__c786nvq22e861n2s1k8i0f28h2ulqhvpg45db7iua25600n0dn9886n8c389eq2p/family1/f813a4a0e0c64ff88a70398538f1daff into d12810111afaf6de0915ac2e031e6b48/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/family1/ced9dab4c039457e94224861eab85b5b_SeqId_28_) 2023-03-24 21:55:47,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__c786nvq22e861n2s1k8i0f28h2ulqhvpg45db7iua25600n0dn9886n8c389eq2p/family1/f813a4a0e0c64ff88a70398538f1daff 2023-03-24 21:55:47,015 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230324 table='testMajorCompactionFromAdmin' column='family1' 2023-03-24 21:55:47,016 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:47,018 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032456e68d201c6e4c2da66b79e1bdd19dc1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032456e68d201c6e4c2da66b79e1bdd19dc1 2023-03-24 21:55:47,019 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032499b5c6c888ad464fb4b6574d78fb0b6f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032499b5c6c888ad464fb4b6574d78fb0b6f 2023-03-24 21:55:47,020 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230324aedac4ba5cb047039f9c428e99b5c04f to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230324aedac4ba5cb047039f9c428e99b5c04f 2023-03-24 21:55:47,021 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230324f167074a7c2244a6a5de07702f24642e to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230324f167074a7c2244a6a5de07702f24642e 2023-03-24 21:55:47,021 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-24 21:55:47,021 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230324' 2023-03-24 21:55:47,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230324 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-24 21:55:47,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230324 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-24 21:55:48,018 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303244777f951456a495e9c0d1277b384abd7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303244777f951456a495e9c0d1277b384abd7 2023-03-24 21:55:48,019 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230324 table='testMajorCompactionFromAdmin' column='family1' 2023-03-24 21:55:48,074 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230324/family1/18bf729a8b484b1fbc1d9e8516cfd0e1 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-24 21:55:48,075 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24., hostname=jenkins-hbase9.apache.org,41807,1679694819103, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230324/family1/18bf729a8b484b1fbc1d9e8516cfd0e1}] 2023-03-24 21:55:48,079 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(763): Validating hfile at hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230324/family1/18bf729a8b484b1fbc1d9e8516cfd0e1 for inclusion in 845f335cfe6009956f8aee7e54864e24/family1 2023-03-24 21:55:48,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-24 21:55:48,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-24 21:55:48,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegion(2446): Flush status journal for 845f335cfe6009956f8aee7e54864e24: 2023-03-24 21:55:48,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230324/family1/18bf729a8b484b1fbc1d9e8516cfd0e1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__ue0lqse17hld74m0inh4obaij1fan7gb523qsnq04f9s3pbk3he994k0uu1sgsl0/family1/18bf729a8b484b1fbc1d9e8516cfd0e1 2023-03-24 21:55:48,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__ue0lqse17hld74m0inh4obaij1fan7gb523qsnq04f9s3pbk3he994k0uu1sgsl0/family1/18bf729a8b484b1fbc1d9e8516cfd0e1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/ceccbed4662d48bf9127953a2471de2b_SeqId_28_ 2023-03-24 21:55:48,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(849): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__ue0lqse17hld74m0inh4obaij1fan7gb523qsnq04f9s3pbk3he994k0uu1sgsl0/family1/18bf729a8b484b1fbc1d9e8516cfd0e1 into 845f335cfe6009956f8aee7e54864e24/family1 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/ceccbed4662d48bf9127953a2471de2b_SeqId_28_ - updating store file list. 2023-03-24 21:55:48,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(882): Loaded HFile hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/ceccbed4662d48bf9127953a2471de2b_SeqId_28_ into 845f335cfe6009956f8aee7e54864e24/family1 2023-03-24 21:55:48,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.HStore(855): Successfully loaded hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__ue0lqse17hld74m0inh4obaij1fan7gb523qsnq04f9s3pbk3he994k0uu1sgsl0/family1/18bf729a8b484b1fbc1d9e8516cfd0e1 into 845f335cfe6009956f8aee7e54864e24/family1 (new location: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/family1/ceccbed4662d48bf9127953a2471de2b_SeqId_28_) 2023-03-24 21:55:48,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41807] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/staging/jenkins__testMajorCompactionFromAdmin__ue0lqse17hld74m0inh4obaij1fan7gb523qsnq04f9s3pbk3he994k0uu1sgsl0/family1/18bf729a8b484b1fbc1d9e8516cfd0e1 2023-03-24 21:55:48,142 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230324 table='testMajorCompactionFromAdmin' column='family1' 2023-03-24 21:55:48,142 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:48,150 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230324042e4fbe8cb541cfb94b0ba50ace3ae1 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230324042e4fbe8cb541cfb94b0ba50ace3ae1 2023-03-24 21:55:48,151 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230324721f8b69893f45c0b73511061d0498d0 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230324721f8b69893f45c0b73511061d0498d0 2023-03-24 21:55:48,152 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023032483bdba911e76466d8dfca85a677433cc to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023032483bdba911e76466d8dfca85a677433cc 2023-03-24 21:55:48,154 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230324b8e8f9ee128e482e839e28bfeb0521b7 to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230324b8e8f9ee128e482e839e28bfeb0521b7 2023-03-24 21:55:48,154 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-24 21:55:48,154 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230324' 2023-03-24 21:55:48,154 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230324 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-24 21:55:48,155 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-24 21:55:48,155 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-24 21:55:48,155 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(367): [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230324c709061c76294512a1ca4c70781af3cc_del] 2023-03-24 21:55:48,155 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:48,158 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230324c709061c76294512a1ca4c70781af3cc_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230324c709061c76294512a1ca4c70781af3cc_del 2023-03-24 21:55:48,158 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-24 21:55:48,158 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(367): [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032417e28ab03198401c9efb790d54175546_del] 2023-03-24 21:55:48,158 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:48,160 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032417e28ab03198401c9efb790d54175546_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032417e28ab03198401c9efb790d54175546_del 2023-03-24 21:55:48,160 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-24 21:55:48,160 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] compactions.PartitionedMobCompactor(367): [hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303246323c5feef354ef8aec1f579cee77db6_del] 2023-03-24 21:55:48,161 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] backup.HFileArchiver(360): Archiving compacted files. 2023-03-24 21:55:48,162 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303246323c5feef354ef8aec1f579cee77db6_del to hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303246323c5feef354ef8aec1f579cee77db6_del 2023-03-24 21:55:48,162 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-24 21:55:48,162 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1679694821891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-24 21:55:48,164 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-24 21:55:48,166 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 2.2840 sec 2023-03-24 21:55:48,185 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:48,188 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:48,205 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-24 21:55:48,206 DEBUG [Listener at localhost/39907] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-24 21:55:48,214 INFO [Listener at localhost/39907] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-24 21:55:48,273 INFO [Listener at localhost/39907] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=255 (was 248) - Thread LEAK? -, OpenFileDescriptor=657 (was 633) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=447 (was 395) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=7832 (was 7849) 2023-03-24 21:55:48,274 DEBUG [Listener at localhost/39907] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33fc1e18 to 127.0.0.1:54680 2023-03-24 21:55:48,275 DEBUG [Listener at localhost/39907] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:48,275 INFO [Listener at localhost/39907] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-03-24 21:55:48,275 INFO [Listener at localhost/39907] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:55:48,275 DEBUG [Listener at localhost/39907] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0672b468 to 127.0.0.1:54680 2023-03-24 21:55:48,275 DEBUG [Listener at localhost/39907] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:48,275 DEBUG [Listener at localhost/39907] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-03-24 21:55:48,275 DEBUG [Listener at localhost/39907] util.JVMClusterUtil(257): Found active master hash=1549481926, stopped=false 2023-03-24 21:55:48,275 INFO [Listener at localhost/39907] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase9.apache.org,46449,1679694817811 2023-03-24 21:55:48,277 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-24 21:55:48,277 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-24 21:55:48,277 INFO [Listener at localhost/39907] procedure2.ProcedureExecutor(629): Stopping 2023-03-24 21:55:48,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-24 21:55:48,277 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-24 21:55:48,277 DEBUG [Listener at localhost/39907] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5ab37b78 to 127.0.0.1:54680 2023-03-24 21:55:48,279 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-24 21:55:48,279 DEBUG [Listener at localhost/39907] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:48,279 INFO [Listener at localhost/39907] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase9.apache.org,41807,1679694819103' ***** 2023-03-24 21:55:48,279 INFO [Listener at localhost/39907] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-03-24 21:55:48,279 INFO [RS:0;jenkins-hbase9:41807] regionserver.HeapMemoryManager(220): Stopping 2023-03-24 21:55:48,280 INFO [RS:0;jenkins-hbase9:41807] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-03-24 21:55:48,280 INFO [RS:0;jenkins-hbase9:41807] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-03-24 21:55:48,280 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for a5fe12acb3b1b5a6f77d77af001cfb67 2023-03-24 21:55:48,281 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 46db6a74ea50377062351d8426481e30 2023-03-24 21:55:48,281 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 471fea1dd43c43f9175a965d62ffe2df 2023-03-24 21:55:48,281 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 9329e175b349e4e9722958f8463695ce 2023-03-24 21:55:48,281 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 4467112a96b2d78b9da907dc14725abd 2023-03-24 21:55:48,281 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 845f335cfe6009956f8aee7e54864e24 2023-03-24 21:55:48,281 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for cc01e67bbfbc97fe3b57bdba3619e55a 2023-03-24 21:55:48,281 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 35c7396e04016558d3f104322a61b9d1 2023-03-24 21:55:48,281 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 3080bc8c83bb721d4a0167e2a8e9a57c 2023-03-24 21:55:48,281 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:48,282 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for c1b745780636112ed0b2ce550374a1fe 2023-03-24 21:55:48,282 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 5e6585bbe76c8302dd7ff1c176f5c30b 2023-03-24 21:55:48,282 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 11f0b79484a112e4405e43efbfd3e887 2023-03-24 21:55:48,282 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 2b21dad992cbb40d94473c32040a2d9a 2023-03-24 21:55:48,282 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for d12810111afaf6de0915ac2e031e6b48 2023-03-24 21:55:48,282 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 5850c24bdc8a7800630587b6a7a58fac 2023-03-24 21:55:48,282 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for a7163a0a63017bfa630fabe660e05be6 2023-03-24 21:55:48,282 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for b8b55d7b0784a827760bb4f27b0708da 2023-03-24 21:55:48,282 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:48,282 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 80c7f51f73f3bd5497dc15e88fadefa2 2023-03-24 21:55:48,282 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1145): stopping server jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:48,282 DEBUG [RS:0;jenkins-hbase9:41807] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x66aaf288 to 127.0.0.1:54680 2023-03-24 21:55:48,282 DEBUG [RS:0;jenkins-hbase9:41807] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:48,282 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing a5fe12acb3b1b5a6f77d77af001cfb67, disabling compactions & flushes 2023-03-24 21:55:48,282 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:48,282 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:48,282 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. after waiting 0 ms 2023-03-24 21:55:48,282 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:48,283 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-03-24 21:55:48,287 INFO [RS:0;jenkins-hbase9:41807] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-03-24 21:55:48,287 INFO [RS:0;jenkins-hbase9:41807] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-03-24 21:55:48,287 INFO [RS:0;jenkins-hbase9:41807] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-03-24 21:55:48,288 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-03-24 21:55:48,288 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-03-24 21:55:48,289 DEBUG [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1505): Waiting on 11f0b79484a112e4405e43efbfd3e887, 1588230740, 2b21dad992cbb40d94473c32040a2d9a, 3080bc8c83bb721d4a0167e2a8e9a57c, 35c7396e04016558d3f104322a61b9d1, 4467112a96b2d78b9da907dc14725abd, 46db6a74ea50377062351d8426481e30, 471fea1dd43c43f9175a965d62ffe2df, 5850c24bdc8a7800630587b6a7a58fac, 5e6585bbe76c8302dd7ff1c176f5c30b, 80c7f51f73f3bd5497dc15e88fadefa2, 845f335cfe6009956f8aee7e54864e24, 9329e175b349e4e9722958f8463695ce, a5fe12acb3b1b5a6f77d77af001cfb67, a7163a0a63017bfa630fabe660e05be6, b8b55d7b0784a827760bb4f27b0708da, c1b745780636112ed0b2ce550374a1fe, cc01e67bbfbc97fe3b57bdba3619e55a, d12810111afaf6de0915ac2e031e6b48, db6e510d903e066fe70066967a0093e2, dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:48,289 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-24 21:55:48,289 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-24 21:55:48,289 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-24 21:55:48,289 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-24 21:55:48,289 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-24 21:55:48,289 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-03-24 21:55:48,294 INFO [regionserver/jenkins-hbase9:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-03-24 21:55:48,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/a5fe12acb3b1b5a6f77d77af001cfb67/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-24 21:55:48,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:48,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for a5fe12acb3b1b5a6f77d77af001cfb67: 2023-03-24 21:55:48,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1679694910713.a5fe12acb3b1b5a6f77d77af001cfb67. 2023-03-24 21:55:48,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 46db6a74ea50377062351d8426481e30, disabling compactions & flushes 2023-03-24 21:55:48,297 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:55:48,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:55:48,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. after waiting 0 ms 2023-03-24 21:55:48,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:55:48,301 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithMonthlyPolicy/46db6a74ea50377062351d8426481e30/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-24 21:55:48,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:55:48,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 46db6a74ea50377062351d8426481e30: 2023-03-24 21:55:48,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1679694832458.46db6a74ea50377062351d8426481e30. 2023-03-24 21:55:48,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 471fea1dd43c43f9175a965d62ffe2df, disabling compactions & flushes 2023-03-24 21:55:48,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:55:48,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:55:48,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. after waiting 0 ms 2023-03-24 21:55:48,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:55:48,334 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/.tmp/info/02c5dd2aee5e481dbc7c963aaad78fa6 2023-03-24 21:55:48,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/471fea1dd43c43f9175a965d62ffe2df/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-24 21:55:48,340 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-03-24 21:55:48,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:55:48,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 471fea1dd43c43f9175a965d62ffe2df: 2023-03-24 21:55:48,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679694861738.471fea1dd43c43f9175a965d62ffe2df. 2023-03-24 21:55:48,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 9329e175b349e4e9722958f8463695ce, disabling compactions & flushes 2023-03-24 21:55:48,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:55:48,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:55:48,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. after waiting 0 ms 2023-03-24 21:55:48,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:55:48,350 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 02c5dd2aee5e481dbc7c963aaad78fa6 2023-03-24 21:55:48,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/9329e175b349e4e9722958f8463695ce/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-24 21:55:48,360 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:55:48,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 9329e175b349e4e9722958f8463695ce: 2023-03-24 21:55:48,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679694871752.9329e175b349e4e9722958f8463695ce. 2023-03-24 21:55:48,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 4467112a96b2d78b9da907dc14725abd, disabling compactions & flushes 2023-03-24 21:55:48,361 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:55:48,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:55:48,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. after waiting 0 ms 2023-03-24 21:55:48,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:55:48,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/4467112a96b2d78b9da907dc14725abd/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-24 21:55:48,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:55:48,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 4467112a96b2d78b9da907dc14725abd: 2023-03-24 21:55:48,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1679694852079.4467112a96b2d78b9da907dc14725abd. 2023-03-24 21:55:48,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 845f335cfe6009956f8aee7e54864e24, disabling compactions & flushes 2023-03-24 21:55:48,377 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:48,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:48,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. after waiting 0 ms 2023-03-24 21:55:48,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:48,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/845f335cfe6009956f8aee7e54864e24/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-24 21:55:48,393 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:48,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 845f335cfe6009956f8aee7e54864e24: 2023-03-24 21:55:48,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1679694934351.845f335cfe6009956f8aee7e54864e24. 2023-03-24 21:55:48,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing cc01e67bbfbc97fe3b57bdba3619e55a, disabling compactions & flushes 2023-03-24 21:55:48,393 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:48,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:48,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. after waiting 0 ms 2023-03-24 21:55:48,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:48,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/cc01e67bbfbc97fe3b57bdba3619e55a/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-24 21:55:48,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:48,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for cc01e67bbfbc97fe3b57bdba3619e55a: 2023-03-24 21:55:48,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1679694934351.cc01e67bbfbc97fe3b57bdba3619e55a. 2023-03-24 21:55:48,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 35c7396e04016558d3f104322a61b9d1, disabling compactions & flushes 2023-03-24 21:55:48,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. 2023-03-24 21:55:48,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. 2023-03-24 21:55:48,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. after waiting 0 ms 2023-03-24 21:55:48,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. 2023-03-24 21:55:48,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2745): Flushing 35c7396e04016558d3f104322a61b9d1 1/1 column families, dataSize=109 B heapSize=592 B 2023-03-24 21:55:48,489 DEBUG [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1505): Waiting on 11f0b79484a112e4405e43efbfd3e887, 1588230740, 2b21dad992cbb40d94473c32040a2d9a, 3080bc8c83bb721d4a0167e2a8e9a57c, 35c7396e04016558d3f104322a61b9d1, 5850c24bdc8a7800630587b6a7a58fac, 5e6585bbe76c8302dd7ff1c176f5c30b, 80c7f51f73f3bd5497dc15e88fadefa2, a7163a0a63017bfa630fabe660e05be6, b8b55d7b0784a827760bb4f27b0708da, c1b745780636112ed0b2ce550374a1fe, d12810111afaf6de0915ac2e031e6b48, db6e510d903e066fe70066967a0093e2, dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:48,689 DEBUG [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1505): Waiting on 11f0b79484a112e4405e43efbfd3e887, 1588230740, 2b21dad992cbb40d94473c32040a2d9a, 3080bc8c83bb721d4a0167e2a8e9a57c, 35c7396e04016558d3f104322a61b9d1, 5850c24bdc8a7800630587b6a7a58fac, 5e6585bbe76c8302dd7ff1c176f5c30b, 80c7f51f73f3bd5497dc15e88fadefa2, a7163a0a63017bfa630fabe660e05be6, b8b55d7b0784a827760bb4f27b0708da, c1b745780636112ed0b2ce550374a1fe, d12810111afaf6de0915ac2e031e6b48, db6e510d903e066fe70066967a0093e2, dd7d45dee3d2e56a7d8a4ae56820f04d 2023-03-24 21:55:48,778 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/.tmp/rep_barrier/e4c5e79c0db844a2a3e02f9df2ad3faa 2023-03-24 21:55:48,786 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e4c5e79c0db844a2a3e02f9df2ad3faa 2023-03-24 21:55:48,799 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/.tmp/table/f5919badf0c7455aa8e42f2143945c40 2023-03-24 21:55:48,803 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f5919badf0c7455aa8e42f2143945c40 2023-03-24 21:55:48,804 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/.tmp/info/02c5dd2aee5e481dbc7c963aaad78fa6 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/info/02c5dd2aee5e481dbc7c963aaad78fa6 2023-03-24 21:55:48,832 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 02c5dd2aee5e481dbc7c963aaad78fa6 2023-03-24 21:55:48,832 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1/.tmp/info/57138e9ffc4846918bb5282c0a1da4d6 2023-03-24 21:55:48,833 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/info/02c5dd2aee5e481dbc7c963aaad78fa6, entries=226, sequenceid=130, filesize=37.9 K 2023-03-24 21:55:48,834 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/.tmp/rep_barrier/e4c5e79c0db844a2a3e02f9df2ad3faa as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/rep_barrier/e4c5e79c0db844a2a3e02f9df2ad3faa 2023-03-24 21:55:48,835 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-24 21:55:48,839 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1/.tmp/info/57138e9ffc4846918bb5282c0a1da4d6 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1/info/57138e9ffc4846918bb5282c0a1da4d6 2023-03-24 21:55:48,839 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e4c5e79c0db844a2a3e02f9df2ad3faa 2023-03-24 21:55:48,839 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4f0c350d to 127.0.0.1:54680 2023-03-24 21:55:48,839 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/rep_barrier/e4c5e79c0db844a2a3e02f9df2ad3faa, entries=1, sequenceid=130, filesize=5.0 K 2023-03-24 21:55:48,840 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:48,843 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/.tmp/table/f5919badf0c7455aa8e42f2143945c40 as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/table/f5919badf0c7455aa8e42f2143945c40 2023-03-24 21:55:48,845 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1/info/57138e9ffc4846918bb5282c0a1da4d6, entries=3, sequenceid=7, filesize=4.8 K 2023-03-24 21:55:48,846 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 35c7396e04016558d3f104322a61b9d1 in 430ms, sequenceid=7, compaction requested=false 2023-03-24 21:55:48,849 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f5919badf0c7455aa8e42f2143945c40 2023-03-24 21:55:48,850 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/table/f5919badf0c7455aa8e42f2143945c40, entries=27, sequenceid=130, filesize=6.6 K 2023-03-24 21:55:48,852 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 562ms, sequenceid=130, compaction requested=false 2023-03-24 21:55:48,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/namespace/35c7396e04016558d3f104322a61b9d1/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-03-24 21:55:48,855 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. 2023-03-24 21:55:48,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 35c7396e04016558d3f104322a61b9d1: 2023-03-24 21:55:48,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1679694821265.35c7396e04016558d3f104322a61b9d1. 2023-03-24 21:55:48,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 3080bc8c83bb721d4a0167e2a8e9a57c, disabling compactions & flushes 2023-03-24 21:55:48,855 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:55:48,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:55:48,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. after waiting 0 ms 2023-03-24 21:55:48,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:55:48,858 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-03-24 21:55:48,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/3080bc8c83bb721d4a0167e2a8e9a57c/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-24 21:55:48,858 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-24 21:55:48,859 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-24 21:55:48,859 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:55:48,859 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-24 21:55:48,859 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 3080bc8c83bb721d4a0167e2a8e9a57c: 2023-03-24 21:55:48,859 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-03-24 21:55:48,859 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1679694852079.3080bc8c83bb721d4a0167e2a8e9a57c. 2023-03-24 21:55:48,859 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing dd7d45dee3d2e56a7d8a4ae56820f04d, disabling compactions & flushes 2023-03-24 21:55:48,860 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:48,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:48,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. after waiting 0 ms 2023-03-24 21:55:48,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:48,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/dd7d45dee3d2e56a7d8a4ae56820f04d/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-24 21:55:48,872 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:48,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for dd7d45dee3d2e56a7d8a4ae56820f04d: 2023-03-24 21:55:48,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1679694891566.dd7d45dee3d2e56a7d8a4ae56820f04d. 2023-03-24 21:55:48,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing c1b745780636112ed0b2ce550374a1fe, disabling compactions & flushes 2023-03-24 21:55:48,873 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:55:48,873 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:55:48,873 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. after waiting 0 ms 2023-03-24 21:55:48,873 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:55:48,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyPolicy/c1b745780636112ed0b2ce550374a1fe/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-24 21:55:48,877 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:55:48,877 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for c1b745780636112ed0b2ce550374a1fe: 2023-03-24 21:55:48,877 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1679694854880.c1b745780636112ed0b2ce550374a1fe. 2023-03-24 21:55:48,877 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 5e6585bbe76c8302dd7ff1c176f5c30b, disabling compactions & flushes 2023-03-24 21:55:48,877 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:48,877 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:48,877 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. after waiting 0 ms 2023-03-24 21:55:48,877 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:48,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5e6585bbe76c8302dd7ff1c176f5c30b/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-24 21:55:48,882 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:48,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 5e6585bbe76c8302dd7ff1c176f5c30b: 2023-03-24 21:55:48,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1679694910713.5e6585bbe76c8302dd7ff1c176f5c30b. 2023-03-24 21:55:48,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 11f0b79484a112e4405e43efbfd3e887, disabling compactions & flushes 2023-03-24 21:55:48,882 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:55:48,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:55:48,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. after waiting 0 ms 2023-03-24 21:55:48,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:55:48,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithWeeklyPolicy/11f0b79484a112e4405e43efbfd3e887/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-24 21:55:48,886 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:55:48,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 11f0b79484a112e4405e43efbfd3e887: 2023-03-24 21:55:48,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1679694840612.11f0b79484a112e4405e43efbfd3e887. 2023-03-24 21:55:48,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 2b21dad992cbb40d94473c32040a2d9a, disabling compactions & flushes 2023-03-24 21:55:48,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:48,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:48,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. after waiting 0 ms 2023-03-24 21:55:48,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:48,890 DEBUG [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1505): Waiting on 2b21dad992cbb40d94473c32040a2d9a, 5850c24bdc8a7800630587b6a7a58fac, 80c7f51f73f3bd5497dc15e88fadefa2, a7163a0a63017bfa630fabe660e05be6, b8b55d7b0784a827760bb4f27b0708da, d12810111afaf6de0915ac2e031e6b48, db6e510d903e066fe70066967a0093e2 2023-03-24 21:55:48,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMinorCompactionWithMonthlyPolicy/2b21dad992cbb40d94473c32040a2d9a/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-24 21:55:48,891 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:48,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 2b21dad992cbb40d94473c32040a2d9a: 2023-03-24 21:55:48,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1679694923805.2b21dad992cbb40d94473c32040a2d9a. 2023-03-24 21:55:48,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing d12810111afaf6de0915ac2e031e6b48, disabling compactions & flushes 2023-03-24 21:55:48,891 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:48,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:48,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. after waiting 0 ms 2023-03-24 21:55:48,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:48,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionFromAdmin/d12810111afaf6de0915ac2e031e6b48/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-24 21:55:48,895 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:48,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for d12810111afaf6de0915ac2e031e6b48: 2023-03-24 21:55:48,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1679694934351.d12810111afaf6de0915ac2e031e6b48. 2023-03-24 21:55:48,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 5850c24bdc8a7800630587b6a7a58fac, disabling compactions & flushes 2023-03-24 21:55:48,895 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:48,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:48,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. after waiting 0 ms 2023-03-24 21:55:48,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:48,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/ns/testMinorCompaction/5850c24bdc8a7800630587b6a7a58fac/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-24 21:55:48,900 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:48,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 5850c24bdc8a7800630587b6a7a58fac: 2023-03-24 21:55:48,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1679694910713.5850c24bdc8a7800630587b6a7a58fac. 2023-03-24 21:55:48,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing a7163a0a63017bfa630fabe660e05be6, disabling compactions & flushes 2023-03-24 21:55:48,900 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:55:48,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:55:48,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. after waiting 0 ms 2023-03-24 21:55:48,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:55:48,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testScannerOnBulkLoadRefHFiles/a7163a0a63017bfa630fabe660e05be6/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-24 21:55:48,904 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:55:48,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for a7163a0a63017bfa630fabe660e05be6: 2023-03-24 21:55:48,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1679694852079.a7163a0a63017bfa630fabe660e05be6. 2023-03-24 21:55:48,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing b8b55d7b0784a827760bb4f27b0708da, disabling compactions & flushes 2023-03-24 21:55:48,904 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:55:48,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:55:48,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. after waiting 0 ms 2023-03-24 21:55:48,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:55:48,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/b8b55d7b0784a827760bb4f27b0708da/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-24 21:55:48,908 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:55:48,908 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for b8b55d7b0784a827760bb4f27b0708da: 2023-03-24 21:55:48,908 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679694822047.b8b55d7b0784a827760bb4f27b0708da. 2023-03-24 21:55:48,908 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing db6e510d903e066fe70066967a0093e2, disabling compactions & flushes 2023-03-24 21:55:48,908 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:48,908 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:48,908 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. after waiting 0 ms 2023-03-24 21:55:48,908 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:48,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/db6e510d903e066fe70066967a0093e2/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-24 21:55:48,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:48,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for db6e510d903e066fe70066967a0093e2: 2023-03-24 21:55:48,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1679694891566.db6e510d903e066fe70066967a0093e2. 2023-03-24 21:55:48,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1604): Closing 80c7f51f73f3bd5497dc15e88fadefa2, disabling compactions & flushes 2023-03-24 21:55:48,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:48,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:48,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. after waiting 0 ms 2023-03-24 21:55:48,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:48,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/data/default/testCompactionWithHFileLink/80c7f51f73f3bd5497dc15e88fadefa2/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-24 21:55:48,917 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:48,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1558): Region close journal for 80c7f51f73f3bd5497dc15e88fadefa2: 2023-03-24 21:55:48,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1679694891566.80c7f51f73f3bd5497dc15e88fadefa2. 2023-03-24 21:55:49,090 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1171): stopping server jenkins-hbase9.apache.org,41807,1679694819103; all regions closed. 2023-03-24 21:55:49,098 DEBUG [RS:0;jenkins-hbase9:41807] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/oldWALs 2023-03-24 21:55:49,099 INFO [RS:0;jenkins-hbase9:41807] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase9.apache.org%2C41807%2C1679694819103.meta:.meta(num 1679694821017) 2023-03-24 21:55:49,101 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/WALs/jenkins-hbase9.apache.org,41807,1679694819103/jenkins-hbase9.apache.org%2C41807%2C1679694819103.1679694820419 not finished, retry = 0 2023-03-24 21:55:49,204 DEBUG [RS:0;jenkins-hbase9:41807] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/oldWALs 2023-03-24 21:55:49,204 INFO [RS:0;jenkins-hbase9:41807] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase9.apache.org%2C41807%2C1679694819103:(num 1679694820419) 2023-03-24 21:55:49,204 DEBUG [RS:0;jenkins-hbase9:41807] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:49,204 INFO [RS:0;jenkins-hbase9:41807] regionserver.LeaseManager(133): Closed leases 2023-03-24 21:55:49,205 INFO [RS:0;jenkins-hbase9:41807] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-03-24 21:55:49,205 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-24 21:55:49,206 INFO [RS:0;jenkins-hbase9:41807] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:41807 2023-03-24 21:55:49,213 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-24 21:55:49,213 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase9.apache.org,41807,1679694819103 2023-03-24 21:55:49,213 ERROR [Listener at localhost/39907-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3eaceaf6 rejected from java.util.concurrent.ThreadPoolExecutor@3a9379fb[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-24 21:55:49,213 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,41807,1679694819103] 2023-03-24 21:55:49,213 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-24 21:55:49,214 ERROR [Listener at localhost/39907-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@17064afe rejected from java.util.concurrent.ThreadPoolExecutor@3a9379fb[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-24 21:55:49,214 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,41807,1679694819103; numProcessing=1 2023-03-24 21:55:49,215 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase9.apache.org,41807,1679694819103 already deleted, retry=false 2023-03-24 21:55:49,215 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,41807,1679694819103 expired; onlineServers=0 2023-03-24 21:55:49,215 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase9.apache.org,46449,1679694817811' ***** 2023-03-24 21:55:49,215 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-03-24 21:55:49,216 DEBUG [M:0;jenkins-hbase9:46449] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d663dfd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-03-24 21:55:49,216 INFO [M:0;jenkins-hbase9:46449] regionserver.HRegionServer(1145): stopping server jenkins-hbase9.apache.org,46449,1679694817811 2023-03-24 21:55:49,216 INFO [M:0;jenkins-hbase9:46449] regionserver.HRegionServer(1171): stopping server jenkins-hbase9.apache.org,46449,1679694817811; all regions closed. 2023-03-24 21:55:49,216 DEBUG [M:0;jenkins-hbase9:46449] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-24 21:55:49,216 DEBUG [M:0;jenkins-hbase9:46449] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-03-24 21:55:49,216 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-03-24 21:55:49,216 DEBUG [M:0;jenkins-hbase9:46449] cleaner.HFileCleaner(317): Stopping file delete threads 2023-03-24 21:55:49,216 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1679694820143] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1679694820143,5,FailOnTimeoutGroup] 2023-03-24 21:55:49,216 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1679694820142] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1679694820142,5,FailOnTimeoutGroup] 2023-03-24 21:55:49,216 INFO [M:0;jenkins-hbase9:46449] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-03-24 21:55:49,216 INFO [M:0;jenkins-hbase9:46449] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-03-24 21:55:49,217 INFO [M:0;jenkins-hbase9:46449] hbase.ChoreService(369): Chore service for: master/jenkins-hbase9:0 had [] on shutdown 2023-03-24 21:55:49,217 DEBUG [M:0;jenkins-hbase9:46449] master.HMaster(1512): Stopping service threads 2023-03-24 21:55:49,217 INFO [M:0;jenkins-hbase9:46449] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-03-24 21:55:49,218 ERROR [M:0;jenkins-hbase9:46449] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-19,5,PEWorkerGroup] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-03-24 21:55:49,218 INFO [M:0;jenkins-hbase9:46449] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-03-24 21:55:49,218 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-03-24 21:55:49,219 DEBUG [M:0;jenkins-hbase9:46449] zookeeper.ZKUtil(398): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-03-24 21:55:49,219 WARN [M:0;jenkins-hbase9:46449] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-03-24 21:55:49,219 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-03-24 21:55:49,219 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-24 21:55:49,219 INFO [M:0;jenkins-hbase9:46449] assignment.AssignmentManager(315): Stopping assignment manager 2023-03-24 21:55:49,219 INFO [M:0;jenkins-hbase9:46449] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-03-24 21:55:49,219 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-24 21:55:49,219 DEBUG [M:0;jenkins-hbase9:46449] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-24 21:55:49,220 INFO [M:0;jenkins-hbase9:46449] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-24 21:55:49,220 DEBUG [M:0;jenkins-hbase9:46449] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-24 21:55:49,220 DEBUG [M:0;jenkins-hbase9:46449] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-24 21:55:49,220 DEBUG [M:0;jenkins-hbase9:46449] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-24 21:55:49,220 INFO [M:0;jenkins-hbase9:46449] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.67 KB heapSize=598.82 KB 2023-03-24 21:55:49,234 INFO [M:0;jenkins-hbase9:46449] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.67 KB at sequenceid=1011 (bloomFilter=true), to=hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/151eb7d8034b4acfb93aa65b6392eb7a 2023-03-24 21:55:49,239 INFO [M:0;jenkins-hbase9:46449] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 151eb7d8034b4acfb93aa65b6392eb7a 2023-03-24 21:55:49,240 DEBUG [M:0;jenkins-hbase9:46449] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/151eb7d8034b4acfb93aa65b6392eb7a as hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/151eb7d8034b4acfb93aa65b6392eb7a 2023-03-24 21:55:49,244 INFO [M:0;jenkins-hbase9:46449] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 151eb7d8034b4acfb93aa65b6392eb7a 2023-03-24 21:55:49,244 INFO [M:0;jenkins-hbase9:46449] regionserver.HStore(1080): Added hdfs://localhost:37101/user/jenkins/test-data/886bf406-0d11-b0a3-f7fb-f5b7305d597b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/151eb7d8034b4acfb93aa65b6392eb7a, entries=123, sequenceid=1011, filesize=33.3 K 2023-03-24 21:55:49,245 INFO [M:0;jenkins-hbase9:46449] regionserver.HRegion(2948): Finished flush of dataSize ~512.67 KB/524976, heapSize ~598.80 KB/613176, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 25ms, sequenceid=1011, compaction requested=false 2023-03-24 21:55:49,246 INFO [M:0;jenkins-hbase9:46449] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-24 21:55:49,246 DEBUG [M:0;jenkins-hbase9:46449] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-24 21:55:49,255 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-24 21:55:49,255 INFO [M:0;jenkins-hbase9:46449] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-03-24 21:55:49,256 INFO [M:0;jenkins-hbase9:46449] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:46449 2023-03-24 21:55:49,263 DEBUG [M:0;jenkins-hbase9:46449] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase9.apache.org,46449,1679694817811 already deleted, retry=false 2023-03-24 21:55:49,314 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-24 21:55:49,314 INFO [RS:0;jenkins-hbase9:41807] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase9.apache.org,41807,1679694819103; zookeeper connection closed. 2023-03-24 21:55:49,314 ERROR [Listener at localhost/39907-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6f590d2c rejected from java.util.concurrent.ThreadPoolExecutor@3a9379fb[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-24 21:55:49,315 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): regionserver:41807-0x1011885c6830001, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-24 21:55:49,315 ERROR [Listener at localhost/39907-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3a4af731 rejected from java.util.concurrent.ThreadPoolExecutor@3a9379fb[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-24 21:55:49,315 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2bbccb29] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2bbccb29 2023-03-24 21:55:49,316 INFO [Listener at localhost/39907] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-03-24 21:55:49,414 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-24 21:55:49,414 INFO [M:0;jenkins-hbase9:46449] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase9.apache.org,46449,1679694817811; zookeeper connection closed. 2023-03-24 21:55:49,414 ERROR [Listener at localhost/39907-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1a191d85 rejected from java.util.concurrent.ThreadPoolExecutor@21cf8947[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 905] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-24 21:55:49,415 DEBUG [Listener at localhost/39907-EventThread] zookeeper.ZKWatcher(600): master:46449-0x1011885c6830000, quorum=127.0.0.1:54680, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-24 21:55:49,415 ERROR [Listener at localhost/39907-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@35340983 rejected from java.util.concurrent.ThreadPoolExecutor@21cf8947[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 905] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-24 21:55:49,417 WARN [Listener at localhost/39907] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-03-24 21:55:49,421 INFO [Listener at localhost/39907] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-24 21:55:49,426 WARN [BP-1477759482-172.31.2.10-1679694814421 heartbeating to localhost/127.0.0.1:37101] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-03-24 21:55:49,426 WARN [BP-1477759482-172.31.2.10-1679694814421 heartbeating to localhost/127.0.0.1:37101] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1477759482-172.31.2.10-1679694814421 (Datanode Uuid 21e86932-ee7d-48f1-8eb1-077868519d45) service to localhost/127.0.0.1:37101 2023-03-24 21:55:49,429 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/cluster_90ef77c7-dcec-9426-4d35-a9af4e7f4c97/dfs/data/data1/current/BP-1477759482-172.31.2.10-1679694814421] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-24 21:55:49,429 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da774604-6ff5-8827-4531-7c56d3e8e5c5/cluster_90ef77c7-dcec-9426-4d35-a9af4e7f4c97/dfs/data/data2/current/BP-1477759482-172.31.2.10-1679694814421] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-24 21:55:49,462 INFO [Listener at localhost/39907] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-24 21:55:49,599 INFO [Listener at localhost/39907] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-03-24 21:55:49,708 INFO [Listener at localhost/39907] hbase.HBaseTestingUtility(1293): Minicluster is down